Estimating SQL Server Table Sizes

Microsoft gives guidance for calculating the estimated sizes of your tables and indexes at http://msdn.microsoft.com/en-us/library/ms175991.aspx.  Performing such estimates is an important step in determining required server capacity.

Unfortunately, Microsoft’s guidance consists of long textual descriptions of the steps that need to be followed to perform the estimations.  It takes some time to interpret the instructions given by Microsoft and perform the calculations.

With that in mind, I put together a spreadsheet to help me perform the calculations.  To use the spreadsheet, I simply fill out the spreadsheet with the total number of rows in the table and details of the columns (data types and sizes) in the table and/or index.  In my experience, the estimates provided by the spreadsheet are within 5-10% of the actual sizes of the tables/indexes, which has been good enough for my needs.

You can download the spreadsheet here.

To complete the spreadsheet, simply enter the requested data into the cells with black borders.  For example, enter the expected number of rows in the table in the cell labeled “Rows in Table’”.  Enter the number of columns in the table in the cell labeled “# of Columns”.  Continue until you have specified all of the information about the table or index.  The spreadsheet uses the information to calculate the total space needed for the table or index.

To help you get started, the spreadsheet includes sample data for a table that contains 1699488 rows and has six columns.  This table has two integer columns (one of which is the primary key), two datetime columns, one tinyint column, and one nvarchar column.  The average size of the data in the nvarchar column is 12 characters.  Compare these table specifications to the sample data in the spreadsheet for an example of how to map the specifications for your table/index into the spreadsheet.

Note that while Microsoft provides guidance for estimating sizes of tables, clustered indexes, and non-clustered indexes, this spreadsheet only handles tables and clustered indexes.  I originally intended to add the calculations for non-clustered indexes, but simply did not find the time.  So, I am providing the spreadsheet as-is.  It should, at the very least, provide a good starting point for estimating table and index sizes.

St. Louis Day of .NET 2011 – Reflections and Session Materials

The annual St. Louis Day of .NET conferences was held August 5th and 6th.  It was the largest yet, with over 800 attendees. 

The keynote address, given by Microsoft’s Jay Schmelzer, focused on Visual Studio Lightswitch.  Lightswitch is billed as a simplified development tool for creating business applications.  I have read comparisons to Microsoft Access, and from what I saw at the keynote, those comparisons are apt.  I’ll give more detail about the keynote and the other sessions that I attended in future posts.

Interestingly, I did not hear anyone talking about Lightswitch after the keynote.  I don’t think I’ve ever been at a conference where there was no further discussion of the keynote.

Among the sessions that I attended were three focusing on MVC… none were great, or even particularly good.  This was very disappointing, as it was a topic of particular interest to me.  On the other hand, sessions on Entity Framework 4.1, SOLID design principles, Powershell, Inversion of Control containers, and jQuery Plug-ins were good.  Two speakers that particularly impressed me were Steve Bohlen (SOLID and Inversion of Control) and Muljadi Budiman(Objective-C).

I found it interesting (but not surprising) that sessions on Objective-C and jQuery were filled to overflowing, despite their focus on non-Microsoft technology and/or vendor-agnostic technologies.  I was happy to see that the developer community, while supportive of Microsoft, isn’t blindly supportive.

One last note… there were a large number of sponsors of this year’s event, and I was impressed by the number that were hiring or looking for contract workers. One hopes that this bodes well for developers looking for work in the St. Louis area.

As far as I know, the conference web site (http://www.stlouisdayofdotnet.com/2011/) doesn’t provide a way for speakers to deposit their session materials.  Because of that, slides and sample code have been slowly trickling out on personal blogs and web sites.  I have been attempting to accumulate as many links to those materials as I can find, and those links are listed below.  Full session descriptions can be found here.

Application Development With HTML5
https://bitbucket.org/bsatrom/applicationdevelopmentwithhtml5/downloads

Architecting Applications the Microsoft Way
http://www.notsotrivial.net/blog/post/2011/08/08/Slides-from-my-St-Louis-Day-of-NET-2011-sessions.aspx

Architecting for Massive Scalability
https://skydrive.live.com/view.aspx?cid=8299A1C60E094D2A&resid=8299A1C60E094D2A%211168

Building jQuery Plugins
https://github.com/irobinson/Presentation-Resources

Building Office 365 Web Parts
http://blog.beckybertram.com/Lists/Posts/Post.aspx?ID=153

Common UX Pitfalls
http://www.slideshare.net/bnunnally/common-ux-pitfalls

Consuming Data From Many Platforms: The Benefits of OData
https://skydrive.live.com/view.aspx?cid=8299A1C60E094D2A&resid=8299A1C60E094D2A%211168

Customizing BlogEngine.NET
http://geekswithblogs.net/scottkuhl/Default.aspx

Design Quality: Learning From the Mistakes of the US Auto Industry
http://www.slideshare.net/jtruemper/design-quality-learning-from-the-mistakes-of-the-us-auto-industry

DotNetNuke Jumpstart
https://github.com/irobinson/Presentation-Resources

Embrace the Buzz – Building Modern DotNetNuke Extensions
https://github.com/irobinson/Presentation-Resources

Get Funcy With C# and F#
https://github.com/dahlbyk/Presentations/tree/master/GetFuncy

IE9 Pinned Sites
http://geekswithblogs.net/scottkuhl/Default.aspx

Introduction to User Experience Methods
http://www.slideshare.net/dgcooley/introduction-to-ux-methods

Javascript is not C#
http://www.codebadger.com/blog/post/2011/08/06/Slides-available-from-2011-Day-of-NET-presentations.aspx

Jumpstart: SharePoint Web Part Development
http://blog.beckybertram.com/Lists/Posts/Post.aspx?ID=153

Keynote: Lightswitch
http://channel9.msdn.com/Blogs/Dan/Jay-Schmelzer-Introducing-Visual-Studio-LightSwitch

Lightswitch Jumpstart
http://www.architectnow.net/Resources.aspx

The “Little Pitfalls” of C#/.NET
http://www.blackrabbitcoder.net/archive/2011/08/11/c.net-little-wonders–pitfalls-the-complete-collection-so-far.aspx

The “Little Wonders” of C#/.NET
http://www.blackrabbitcoder.net/archive/2011/08/11/c.net-little-wonders–pitfalls-the-complete-collection-so-far.aspx

Making $$$ with Windows Phone 7
https://skydrive.live.com/view.aspx?cid=8299A1C60E094D2A&resid=8299A1C60E094D2A%211170

The Making of the Day of .NET Web Site with Orchard CMS
http://www.notsotrivial.net/blog/post/2011/08/08/Slides-from-my-St-Louis-Day-of-NET-2011-sessions.aspx

Overview of Windows Azure Marketplace DataMarket
https://skydrive.live.com/view.aspx?cid=8299A1C60E094D2A&resid=8299A1C60E094D2A%211172

Powershell: Not Your Father’s Command Line
http://blogs.technet.com/b/matthewms/p/powershell.aspx

Reactive Extensions (Rx) in .NET
http://www.architectnow.net/Resources.aspx

Refactoring to a SOLID Foundation
http://unhandled-exceptions.com/blog/index.php/2011/08/11/st-louis-day-of-netthats-a-wrap/

Startup Tips and Tricks
http://www.architectnow.net/Resources.aspx

Taming Your Dependencies With Inversion of Control Containers
http://unhandled-exceptions.com/blog/index.php/2011/08/11/st-louis-day-of-netthats-a-wrap/

The Three Tasty Flavors of Entity Framework 4.1
http://www.codebadger.com/blog/post/2011/08/06/Slides-available-from-2011-Day-of-NET-presentations.aspx

Unit Testing Patterns and Anti-Patterns
http://unhandled-exceptions.com/blog/index.php/2011/08/11/st-louis-day-of-netthats-a-wrap/

What’s Wrong With This Picture?
http://openlightgroup.net/Blog/tabid/58/EntryId/183/Whats-Wrong-with-this-Picture-St-Louis-Day-of-Net-Presentation-Info.aspx

Why You Should Care the Sony PSN Was Hacked
https://github.com/k0emt/Presentations

Updating My Dev Environment: VS2010SP1, MVC3, EF4.1, and NuGet vs the Web Platform Installer.

Earlier this week I realized that I had let the development environment on my laptop get a bit out of date.  Sure, it was running Visual Studio 2010, but I had not applied Service Pack 1, nor any of the big buzzy out-of-band add-ons from the first half of  2011.

So, I sat down to install Visual Studio 2010 Service Pack 1, ASP.NET MVC3, Entity Framework 4.1, and the Nuget package manager.  I learned a few things along the way.  Following is a recap of the upgrade process, along with some notes about what I know now that I didn’t know before.

Step 1) Install Visual Studio 2010 Service Pack 1.  This installation was a painless hands-off procedure, although it did take 2-3 hours to complete.  My dev tools are set up in virtual machines (one for Visual Studio 2010 and related tools, one for WAMP platform tools), so running the upgrade against the virtual machine probably slowed things down.  Regardless, it was a lengthy install.

Step 2) Go to nuget.org and install the NuGet Visual Studio extension.  Again, this was painless; I downloaded the installation package, chose Run, and that was it.  NuGet is a package manager for easily installation and updating of open source libraries and tools in Visual Studio.  Package managers are already popular in other environments, but are a relatively new addition to the Microsoft development stack.

Step 3) Start Visual Studio.  So far, so good.

Here is where things started to go awry.  My own fault, as you will see.  I followed the instructions found at http://docs.nuget.org/docs/start-here/Using-the-Package-Manager-Console to attempt to install the latest versions of ASP.NET MVC and Entity Framework.  I could have just done this via the usual download-and-install process, but I wanted to give NuGet a spin.

Step 4) From the Tools menu, use NuGet by selecting Library Package Manager and then clicking Package Manager Console.  This opens a PowerShell dialog within Visual Studio.

Step 5) Type "get-help NuGet" to view all of the available NuGet commands.

Step 6) Type "Get-Package -ListAvailable" to view a list of all available NuGet packages.  Locate the “Entity Framework” package in the list.

Step 7) Type "Install-Package Entity Framework".  Oops… FAIL. I received the following error:

    Install-Package : The current environment doesn’t have a solution open.
    At line:1 char:16
    + Install-Package <<<<  EntityFramework
        + CategoryInfo          : InvalidOperation: (:) [Install-Package], InvalidOperationException
        + FullyQualifiedErrorId : NuGetNoActiveSolution,NuGet.PowerShell.Commands.InstallPackageCommand 

What I’d uncovered was a fundamental misunderstanding on my part about what NuGet does.  My desire was to add capabilities to Visual Studio itself, not just to a single project or solution.  Turns out that’s not what NuGet is for. 

Step 8 ) Type "get-help Install-Package" to get more information about what I had just attempted.  Included in the description of the “Install-Package” command is the statement "Installs a package and its dependencies into the project."  Ok then, that confirms it… NuGet is for individual projects, not for the development environment as a whole.

My initial misunderstanding of NuGet stems from my only previous Package Manager experience, which was with the “apt” system that is a part of Ubuntu Linux..  Using that Package Manager, I was able to add capabilities and tools to the operating system… things like MySQL and Apache.  This, in fact, is somewhat similar to the Web Platform Installer from Microsoft.  But not NuGet.  NuGet is actually more like RubyGems, which are used to adding capabilities to software projects written in Ruby. 

So, I learned some useful information, but as far as installing ASP.NET MVC and Entity Framework, it was back to the drawing board

Step 9) Shut down Visual Studio.

Step 10) Install ASP.NET MVC 3.  Speaking of the Web Platform Installer, I hadn’t used it a while, so I decided to go that route to install ASP.NET MVC 3.  I browsed to http://www.microsoft.com/web/gallery/install.aspx?appid=MVC3, and clicked Install Now.  The Web Platform Installer itself installed first, followed by ASP.NET MVC 3.  This took much longer than expected to install.  Overall, though, Web Platform Installer was as painless to use as I remembered.  Nice.

Step 11) Install IIS Express 7.5.  Though not part of the original plan, I decided to also install IIS Express 7.5.  I did this using a "normal" download-and-install procedure.  Easy.  As advertised, very lightweight.

Step 12) Install Entity Framework 4.1.  I did this via the download-and-install process (choosing to "save" and then run the downloadable installer… Web Platform Installer was not used).  This installation was very quick when compared to MVC3.  I’m not sure if that was due to the NON-use of Web Platform Installer to install EF, or if EF is simply a much more lightweight install than MVC.

While looking for the correct Entity Framework 4 installer package I noticed the following statement on the ADO.NET blog: "Note: The NuGet package only includes the EF 4.1 runtime and does not include the Visual Studio item templates for using DbContext with Model First and Database First development." So, that provides further clarification that NuGet adds the necessary run-time libraries to specific projects, but does NOT add full development tools to VS.

Step 13) Start Visual Studio and verify all installs.  To confirm that everything was as it should be, I checked the following:

    Step 13a) File/New/Project shows ASP.NET MVC3 Web Application as a project type.
    Step 13b) EntityFramework 4.1.0.0 shows up as an option when adding a reference to a project.
    Step 13c) There is a new option for Add Library Package Reference when adding a reference to a project.
    Step 13d) On the property page for a a web project, there is a new option for running under IIS Express.

That’s it.  My Visual Studio 2010 development environment is now up-to-date. Along the way, I gained greater insight into NuGet, the hot new Package Manager add-on for Visual Studio  In particular, I learned that if you’re looking to upgrade your entire development environment, and want to use a package-manager type application, Web Platform Installer is your choice.  On the other hand, if you want to add libraries to a project on which you are working, Nuget is the tool to use.

Open Letter to Microsoft Regarding the Skype Acquisition

When I started this blog, I decided that I’d blog about technologies, tools, and gadgets, but would try to avoid straight opinion pieces.  Well, I’m going to break that rule.

Like many others, I was surprised by the recent announcement that Microsoft is acquiring Skype.

I won’t pretend to understand the business and technical strategies that drove this deal, or what the implications might be.  Check out the usual tech news outlets and you’ll find that there are plenty of others doing just that.  The possibilities of the deal are exciting, and I look forward to witnessing the outcome.  But I’m not going to try and guess the outcome in advance..

A recent post to Twitter reads “Wonder what the #MS acquisition of #Skype means for its cross-platform availability.”  That got me to thinking.  Currently, Skype clients are available on many platforms, including Windows, OSX, Linux, iPhone, Android-based phones, and Symbian phones.  There are even Skype-enabled televisions.  Does an acquisition by Microsoft put this broad platform reach in jeopardy?

I work on a project that includes partners spread across the United States, England, Germany, Austria, France, Eqypt, China, Australia, Brazil, and probably a few that I’ve forgotten.  These partners use a huge variety of technologies and platforms.  Off the top of my head, I can think of Windows, OSX, and Linux operating systems.  MySql, Microsoft SQL Server, and PostgreSQL database servers.  IIS, Apache, and NGINX web servers.  Programming languages in use include C#, Java, and PHP (I’m sure there are others).  Additional technologies in use include Drupal, Fedora Commons, Gluster, and many more.  With this number of partners and technologies spread across the world, it should be no surprise that there is no consensus on what are the “best” tools.  Each partner uses the tools that work best for them.

Similarly, each partner institution (and in some cases each individual person), initially had different preferences for instant communication tools.  Some preferred Windows Live Messenger, some liked iChat.  There were Google Chat advocates, and even some Yahoo! Messenger users.  Some of these tools are single-platform, and many do not talk to one another. 

So how did we ultimately find a way to communicate with one another?  Skype.  The integration of voice, video, and chat was compelling.  The ability to call someone in another country for free was significant.  And, the variety of platforms supported allowed ALL of our partners to use the tool, regardless of their preferred computing platform.

Skype has become an invaluable tool.  We’ve come to rely on it so much that we were negatively affected earlier this year when Skype suffered a major outage.

So, Microsoft, I’m sure you see the various platforms and partners as an opportunity to sell more customers on the Windows platform.  Nothing wrong with that; it’s your job to find opportunities to push your products.  Except the opportunity you imagine doesn’t exist.  This is NOT an opportunity to push Windows.  No partner is going to switch platforms simply to use an instant communications tool, especially when other options exist, even if those options are technically inferior.

We can’t be the only existing Skype users in this situation.  Please, Microsoft, keep supporting ALL of the platforms on which Skype exists.  I was at the MIX conference where the first version of Silverlight was announced, and I remember Scott Guthrie saying “this was HARD” when Silverlight 1.0 running on OSX was demonstrated.  So cross-platform is hard, but you did it.  Do it again.

If cross-platform availability is easier to achieve by dropping the platform-specific client applications in favor of a slick web-based cross-browser cross-platform Skype application, so be it.  Actually, that would be great.  Just make sure it really IS cross-platform and cross-browser.  Not Silverlight-based, and none of this “HTML5 runs best in Internet Explorer on Windows” silliness.  Ensure that Skype continues to work the same everywhere that it does now.

C’mon Microsoft, you can do it.  Skype is a fantastic tool.  Keep it that way, and keep it cross-platform.

Log Parser Rocks! More than 50 Examples!

Log Parser is a tool that has been around for quite some time (almost six years, in fact).  I can’t really do any better than the description on the official download page, so here it is: “Log parser is a powerful, versatile tool that provides universal query access to text-based data such as log files, XML files and CSV files, as well as key data sources on the Windows operating system such as the Event Log, the Registry, the file system, and Active Directory”.  

Log Parser is a command line (yes, command line!) tool that uses a SQL dialect to extract information from data sources.  In particular, I have found it to be invaluable for extracting information from the web server logs of the sites that I manage and develop.

First, about that SQL syntax Log Parser uses to query the data sources… many developers seem to have a natural aversion to SQL.  In addition, many new data access frameworks attempt to abstract SQL away from the developer.  However, I have always found SQL easy to work with and believe it to be an essential tool that every developer should at least have a working knowledge of.   For Log Parser, all that is necessary is a basic understanding of the core SQL SELECT statement, as implemented within Microsoft’s SQL Server (that is, T-SQL).  That means you should be familiar with the following elements of a SELECT statement: TOP, FROM, INTO, WHERE, ORDER BY, GROUP BY.  That’s all you need to perform most Log Parser operations.

Curiously, Log Parser has never received the amount of attention that I think it deserves.  Beyond a flurry of attention when it was first released, it seems to be mentioned rarely in official Microsoft communications or blogs.  Despite that, it remains a viable and valuable tool for parsing not just web server log files, but all types of structured text-based data.

In this post, rather than explaining how to use Log Parser. I’ll give a number of examples of its use.  In addition, I’ll document some useful locations where Log Parser information can be found on the web.

Examples

Keep in mind that most of the examples that I give here are all-in-one command line queries (even though many wrap to multiple lines when displayed here).  However, queries can also be run as

logparser file:XXXXX.sql

where XXXXX is the name of a file containing a logparser-friendly sql query.  There are a couple examples of this in the following list.

The examples given here have been obtained from a variety of sources, including the documentation that ships with the tool, blogs and online documentation, and my own experience.  Unfortunately, I don’t have a record of the origin of each individual example, as I’ve compiled these piecemeal over the last two or three years.

I hope you’ll find something useful here and gain an appreciation for just how robust this tool is.

1)  All pages hits by a given IP address

logparser "select cs-uri-stem, count(cs-uri-stem) as requestcount from [LogFileName] where c-ip = ‘000.00.00.000’ group by cs-uri-stem order by count(cs-uri-stem) desc"

2) Hits on a particular page by IP address

logparser "select c-ip, count(c-ip) as requestcount from [LogFileName] where cs-uri-stem like ‘/search.aspx%’ group by c-ip order by count(c-ip) desc"

3)  ReverseDNS example.  This attempts to find the domain associated with a given IP address.

logparser "select c-ip, REVERSEDNS(c-ip) from [LogFileName] where c-ip = ‘000.00.00.000’ group by c-ip"

4)  CSV example. All hits on a page, written to a CVS file.

logparser "select * into OUTPUT.CSV from [LogFileName] where cs-uri-stem like ‘/pagename.aspx’"

5)  Chart example.  All hits on a page by an IP address, displayed on a chart.

logparser "select c-ip, count(c-ip) as requestcount into logparserchart.gif from [LogFileName] where cs-uri-stem like ‘/pagename.aspx’ group by c-ip order by count(c-ip) desc" -o:chart

6)  Hits per hour from a particular IP address

logparser "select TO_LOCALTIME(QUANTIZE(TO_TIMESTAMP(date, time), 3600)), count(*) as numberrequests from [LogFileName] where c-ip=’000.000.00.000′ group by TO_LOCALTIME(QUANTIZE(TO_TIMESTAMP(date,time), 3600))"

7)  Basic list of IP addresses generating traffic

logparser "select c-ip, count(c-ip) as requestcount from [LogFileName] group by c-ip order by count(c-ip) desc"

8)  Basic list of pages being hit

logparser "select cs-uri-stem, count(cs-uri-stem) from [LogFileName] where cs-uri-stem like ‘%aspx%’ or cs-uri-stem like ‘%ashx%’ group by cs-uri-stem order by count(cs-uri-stem) desc"

9)  Basic list of pages being hit, including which IPs are doing the hitting

logparser "select cs-uri-stem, c-ip, count(cs-uri-stem) from [LogFileName] where cs-uri-stem like ‘%aspx%’ or cs-uri-stem like ‘%ashx%’ group by cs-uri-stem, c-ip order by count(cs-uri-stem) desc"

10)  Pages being hit after a specific date and time

logparser "select cs-uri-stem, c-ip, count(cs-uri-stem) from [LogFileName] where cs-uri-stem like ‘%aspx%’ or cs-uri-stem like ‘%ashx%’ and date=’2009-06-04′ and time > ’15:00:00′ group by cs-uri-stem, c-ip order by count(cs-uri-stem) desc"

11)  Counts of hits of ASPX/ASHX pages by hour from a particular IP address

logparser "select TO_LOCALTIME(QUANTIZE(TO_TIMESTAMP(date, time), 3600)), count(*) as numberrequests from [LogFileName] where c-ip=’000.000.00.00′ and (cs-uri-stem like ‘%aspx%’ or cs-uri-stem like ‘%ashx%’) group by TO_LOCALTIME(QUANTIZE(TO_TIMESTAMP(date,time), 3600))"

12)  Counts of hits against specific pages by hour from a particular IP address

logparser "select TO_LOCALTIME(QUANTIZE(TO_TIMESTAMP(date, time), 3600)), cs-uri-stem, count(*) as numberrequests from [LogFileName] where c-ip=’000.000.00.00′ and (cs-uri-stem like ‘%aspx%’ or cs-uri-stem like ‘%ashx%’) group by TO_LOCALTIME(QUANTIZE(TO_TIMESTAMP(date,time), 3600)), cs-uri-stem order by numberrequests desc"

13)  Top browsers

logparser "Select top 50 to_int(mul(100.0,PropCount(*))) as Percent, count(*) as TotalHits, cs(User-Agent) as Browser from [LogFileName] group by Browser order by Totalhits desc"

14)  Hourly Bandwidth (chart)

logparser "Select TO_LOCALTIME(QUANTIZE(TO_TIMESTAMP(date, time), 3600)) As Hour, Div(Sum(cs-bytes),1024) As Incoming(K), Div(Sum(sc-bytes),1024) As Outgoing(K) Into BandwidthByHour.gif From [LogFileName] Group By Hour"

15)  Requests by URI

logparser "SELECT top 80 QUANTIZE(TO_TIMESTAMP(date, time), 3600) as Hour, TO_LOWERCASE(STRCAT(‘/’,EXTRACT_TOKEN(cs-uri-stem,1,’/’))) as URI, COUNT(*) AS RequestsPerHour, SUM(sc-bytes) AS TotBytesSent, AVG(sc-bytes) AS AvgBytesSent, Max(sc-bytes) AS MaxBytesSent, ADD(1,DIV(Avg(time-taken),1000)) AS AvgTime, ADD(1,DIV(MAX(time-taken),1000)) AS MaxTime FROM [LogFileName] GROUP BY Hour, URI Having RequestsPerHour > 10 ORDER BY RequestsPerHour ASC"

16)  Top 10 Images by size

logparser "Select Top 10 StrCat(Extract_Path(TO_Lowercase(cs-uri-stem)),’/’) AS RequestedPath, Extract_filename(To_Lowercase(cs-uri-stem)) As RequestedFile, Count(*) AS Hits, Max(time-taken) As MaxTime, Avg(time-taken) As AvgTime, Max(sc-bytes) As BytesSent From [LogFileName] Where (Extract_Extension(To_Lowercase(cs-uri-stem)) IN (‘gif';’jpg';’png’)) AND (sc-status = 200) Group By To_Lowercase(cs-uri-stem) Order By BytesSent, Hits, MaxTime DESC"

17)  Top 10 URLs for a website, with total hits, max time to serve, and average time to serve

logparser "Select TOP 10 STRCAT(EXTRACT_PATH(cs-uri-stem),’/’) AS RequestPath, EXTRACT_FILENAME(cs-uri-stem) AS RequestedFile, COUNT(*) AS TotalHits, Max(time-taken) AS MaxTime, AVG(time-taken) AS AvgTime, AVG(sc-bytes) AS AvgBytesSent FROM [LogFileName] GROUP BY cs-uri-stem ORDER BY TotalHits DESC"

18)  Top 20 clients

logparser "Select Top 20 c-ip AS Client, Count(*) AS Hits INTO Chart.gif FROM [LogFileName] GROUP BY c-ip ORDER BY Hits Desc"

19)  Referrer Broken Links (i.e. external references to broken links on your site)

logparser "SELECT DISTINCT cs(Referer) as Referer, cs-uri-stem as Url INTO ReferBrokenLinks.html FROM [LogFileName] WHERE cs(Referer) IS NOT NULL AND sc-status = 404 AND (sc-substatus IS NULL OR sc-substatus=0)" -tpl:ReferBrokenLinks.tpl

20)  Status codes

logparser "SELECT sc-status As Status, COUNT(*) As Number INTO StatusCodes.gif FROM <2> GROUP BY Status ORDER BY Status"

21)  Search the Event Log for W3SVC (IIS) log entries and color-coordinate as to Error, Warning, Information.  This example writes the output of the query to an HTML file that  is generated using a template file.

logparser "SELECT TimeGenerated,EventTypeName,Strings,Message,CASE EventTypeName WHEN ‘Error event’ THEN ‘RED’ WHEN ‘Warning event’ THEN ‘YELLOW’ WHEN ‘Information event’ THEN ‘WHITE’ ELSE ‘BLUE’ END As Color INTO file.html FROM System WHERE SourceName = ‘W3SVC’"  -tpl:IISEventLogEntries.tpl

Where IISEventLogEntries.tpl is a file that contains the following:

<LPHEADER>
<HTML>
<HEAD>
  <STYLE>
    TD { font-family: Arial };
    TH { font-family: Arial };
  </STYLE>
</HEAD>
<BODY>
<TABLE BORDERCOLOR="BLACK" BORDER="1" CELLPADDING="2" CELLSPACING="2">
<TR>
  <TH COLSPAN=4 BGCOLOR="BLACK"><FONT COLOR=WHITE>New W3SVC Messages in System Event Log</FONT></TH>
</TR>
<TR>
  <TH ALIGN=LEFT BGCOLOR="#C0C0C0">Time Generated</TH>
  <TH ALIGN=LEFT BGCOLOR="#C0C0C0">Event Type</TH>
  <TH ALIGN=LEFT BGCOLOR="#C0C0C0">Strings</TH>
  <TH ALIGN=LEFT BGCOLOR="#C0C0C0">Message</TH>
</TR>
</LPHEADER>
<LPBODY>
<TR bgCOLOR="%Color%">
  <TD>%TimeGenerated%</TD>
  <TD>%EventTypeName%</TD>
  <TD>%Strings%</TD>
  <TD>%Message%</TD>
</TR>
</LPBODY>
</TABLE>
</BODY>
</HTML>

22)  Upload Log Parser query results directly to a table in SQL Server

logparser "select * into LogTable from [LogFileName] where cs-uri-stem like ‘/folder/filename%’" -o:SQL -createTable:ON -server:[DatabaseServer] -database:[Database] -username:[SqlUser] -password:[SqlPassword]

23)  Top 10 images by size sent.  Note that this example also shows how to query multiple log files at once.

logparser "Select Top 10 StrCat(Extract_Path(TO_Lowercase(cs-uri-stem)),’/’) AS RequestedPath, Extract_filename(To_Lowercase(cs-uri-stem)) As RequestedFile, Count(*) AS Hits, Max(time-taken) As MaxTime, Avg(time-taken) As AvgTime, Max(sc-bytes) As BytesSent INTO TOP10ImagesBySize.txt FROM logs\iis\ex*.log WHERE (Extract_Extension(To_Lowercase(cs-uri-stem)) IN  (‘gif';’jpg';’png’)) AND (sc-status = 200) GROUP BY To_Lowercase(cs-uri-stem) ORDER BY BytesSent, Hits, MaxTime DESC"

24)  Browser types (two different approaches)

logparser "SELECT distinct cs(User-Agent), count(*) as hits INTO useragentsalltypes.txt FROM logs\iis\ex*.log GROUP BY cs(user-agent) ORDER BY hits DESC"

logparser "SELECT TO_INT(MUL(100.0,PROPCOUNT(*))) AS Percent,  COUNT(*) AS Hits, cs(User-Agent) as Browser INTO  UseragentsHits.txt FROM  logs\iis\ex*.log  GROUP BY Browser ORDER BY HITS DESC"

25)  Unique visitors per day.  This requires two queries.  The first query selects from the IIS logs into a CSV file, and the second selects from that CSV file.

logparser "SELECT DISTINCT cs-username, date INTO tempUniqueVisitorsPerDay.csv FROM logs\iis\ex*.log WHERE cs-username <> NULL Group By Date, cs-username"

logparser "SELECT date, count(cs-username) as UniqueVisitors into test.txt FROM tempUniqueVisitorsPerDay.csv GROUP BY date"

26)  Top 10 largest ASPX pages.

logparser "Select Top 10 StrCat(Extract_Path(TO_Lowercase(cs-uri-stem)),’/’) AS  RequestedPath, Extract_filename(To_Lowercase(cs-uri-stem)) As RequestedFile,  Count(*) AS Hits, Max(time-taken) As MaxTime, Avg(time-taken) As AvgTime, Max(sc-bytes) As BytesSent INTO top10pagesbysize.txt FROM logs\iis\ex*.log WHERE (Extract_Extension(To_Lowercase(cs-uri-stem)) IN (‘aspx’)) AND  (sc-status = 200) GROUP BY To_Lowercase(cs-uri-stem) ORDER BY BytesSent, Hits, MaxTime DESC"

27)  Top 10 slowest ASPX pages

logparser "SELECT TOP 10 cs-uri-stem, max(time-taken) as MaxTime, avg(time-taken) as AvgTime INTO toptimetaken.txt FROM logs\iis\ex*.log WHERE extract_extension(to_lowercase(cs-uri-stem)) = ‘aspx’ GROUP BY cs-uri-stem ORDER BY MaxTime DESC"

28)  Top 10 slowest ASPX pages on a specific day

logparser "SELECT TOP 10 cs-uri-stem, max(time-taken) as MaxTime, avg(time-taken) as AvgTime INTO toptimetaken.txt FROM logs\iis\ex*.log WHERE extract_extension(to_lowercase(cs-uri-stem)) = ‘aspx’ AND TO_STRING(To_timestamp(date, time), ‘MMdd’)=’1003’  GROUP BY cs-uri-stem ORDER BY MaxTime DESC"

29)  Daily bandwidth

logparser "Select To_String(To_timestamp(date, time), ‘MM-dd’) As Day, Div(Sum(cs-bytes),1024) As Incoming(K), Div(Sum(sc-bytes),1024) As Outgoing(K) Into BandwidthByDay.gif From logs\iis\ex*.log Group By Day"

30)  Bandwidth by hour

logparser "SELECT QUANTIZE(TO_TIMESTAMP(date, time), 3600) AS Hour, SUM(sc-bytes) AS TotalBytesSent INTO BytesSentPerHour.gif FROM logs\iis\ex*.log GROUP BY Hour ORDER BY Hour"

31)  Average page load time per user

logparser "Select Top 20 cs-username AS UserName, AVG(time-taken) AS AvgTime,  Count(*) AS Hits INTO AvgTimePerUser.txt FROM logs\iis\ex*.log WHERE cs-username IS NOT NULL GROUP BY cs-username ORDER BY AvgTime DESC"

32)  Ave page load time for a specific user

logparser "Select cs-username AS UserName, AVG(time-taken) AS AvgTime,  Count(*) AS Hits INTO AvgTimeOnSpecificUser.txt FROM logs\iis\ex*.log WHERE cs-username = ‘CONTOSO\User1234’ GROUP BY cs-username"

33)  Error trends.  This query is quite long, and is easier expressed in a text file than on the command line.  So, Log Parser reads and executes the query contained in the specified text file.

logparser file:errortrend.sql

Where errortrend.sql contains the following:

SELECT
  TO_STRING(To_timestamp(date, time), ‘MMdd’) AS Day,
  SUM(c200) AS 200s,
  SUM(c206) AS 206s,
  SUM(c301) AS 301s,
  SUM(c302) AS 302s,
  SUM(c304) AS 304s,
  SUM(c400) AS 400s,
  SUM(c401) AS 401s,
  SUM(c403) AS 403s,
  SUM(c404) AS 404s,
  SUM(c500) AS 500s,
  SUM(c501) AS 501s,
  SUM(c502) AS 502s,
  SUM(c503) AS 503s,
  SUM(c504) AS 504s,
  SUM(c505) AS 505s
USING
  CASE sc-status WHEN 200 THEN 1 ELSE 0 END AS c200,
  CASE sc-status WHEN 206 THEN 1 ELSE 0 END AS c206,
  CASE sc-status WHEN 301 THEN 1 ELSE 0 END AS c301,
  CASE sc-status WHEN 302 THEN 1 ELSE 0 END AS c302,
  CASE sc-status WHEN 304 THEN 1 ELSE 0 END AS c304,
  CASE sc-status WHEN 400 THEN 1 ELSE 0 END AS c400,
  CASE sc-status WHEN 401 THEN 1 ELSE 0 END AS c401,
  CASE sc-status WHEN 403 THEN 1 ELSE 0 END AS c403,
  CASE sc-status WHEN 404 THEN 1 ELSE 0 END AS c404,
  CASE sc-status WHEN 500 THEN 1 ELSE 0 END AS c500,
  CASE sc-status WHEN 501 THEN 1 ELSE 0 END AS c501,
  CASE sc-status WHEN 502 THEN 1 ELSE 0 END AS c502,
  CASE sc-status WHEN 503 THEN 1 ELSE 0 END AS c503,
  CASE sc-status WHEN 504 THEN 1 ELSE 0 END AS c504,
  CASE sc-status WHEN 505 THEN 1 ELSE 0 END AS c505
INTO ErrorChart.gif
FROM
    logs\iis\ex*.log
GROUP BY
  Day
ORDER BY
  Day

34)  Win32 errors

logparser "SELECT sc-win32-status as ErrorNumber, WIN32_ERROR_DESCRIPTION(sc-win32-status) as ErrorDesc, Count(*) AS Total INTO Win32ErrorNumbers.txt FROM logs\iis\ex*.log WHERE sc-win32-status>0 GROUP BY ErrorNumber ORDER BY Total DESC"

35)  Substatus codes

logparser "SELECT sc-status, sc-substatus, Count(*) AS Total INTO 401subcodes.txt FROM logs\iis\ex*.log WHERE sc-status=401 GROUP BY sc-status, sc-substatus ORDER BY sc-status, sc-substatus DESC"

36)  Substatus codes per day.  This is another example of executing a query contained in a text file.

logparser file:substatusperday.sql

Where substatusperday.sql contains the following:

SELECT
  TO_STRING(To_timestamp(date, time), ‘MMdd’) AS Day,
  SUM(c1) AS 4011,
  SUM(c2) AS 4012,
  SUM(c3) AS 4013,
  SUM(c4) AS 4014,
  SUM(c5) AS 4015,
  SUM(c7) AS 4017
USING
  CASE sc-substatus WHEN 1 THEN 1 ELSE 0 END AS c1,
  CASE sc-substatus WHEN 2 THEN 1 ELSE 0 END AS c2,
  CASE sc-substatus WHEN 3 THEN 1 ELSE 0 END AS c3,
  CASE sc-substatus WHEN 4 THEN 1 ELSE 0 END AS c4,
  CASE sc-substatus WHEN 5 THEN 1 ELSE 0 END AS c5,
  CASE sc-substatus WHEN 7 THEN 1 ELSE 0 END AS c7
INTO
  401subcodesperday.txt
FROM
  logs\iis\ex*.log
WHERE
  sc-status=401
GROUP BY
  Day
ORDER BY
  Day

37)  Substatus codes per page

logparser "SELECT TOP 20 cs-uri-stem, sc-status, sc-substatus, Count(*) AS Total INTO 401Pagedetails.txt FROM logs\iis\ex*.log WHERE sc-status=401 GROUP BY cs-uri-stem, sc-status, sc-substatus ORDER BY Total"

38)  MB sent per HTTP status code

logparser "SELECT EXTRACT_EXTENSION(cs-uri-stem) AS PageType, SUM(sc-bytes) as TotalBytesSent, TO_INT(MUL(PROPSUM(sc-bytes), 100.0)) AS PercentBytes INTO PagesWithLargestBytesSent.htm FROM logs\iis\ex*.log GROUP BY Pagetype ORDER BY PercentBytes DESC"

39) 500 errors per ASPX and Domain User

logparser "SELECT cs-username, cs-uri-stem, count(*) as Times INTO 500PagesByUserAndPage.txt FROM logs\iis\ex*.log WHERE sc-status=500 GROUP BY  cs-username, cs-uri-stem ORDER BY Times DESC"

40)  Percent of 500 errors caused by each user

logparser "SELECT cs-username, count(*) as Times, propcount(*) as Percent INTO 500ErrorsByUser.csv FROM  logs\iis\ex*.log WHERE sc-status=500 GROUP BY cs-username ORDER BY Times DESC"

41)  Determine what percentage of the total bytes sent are being caused by each page type

logparser "SELECT EXTRACT_EXTENSION(cs-uri-stem) AS PageType, SUM(sc-bytes) as TotalBytesSent, TO_INT(MUL(PROPSUM(sc-bytes), 100.0)) AS PercentBytes INTO PagesWithLargestBytesSent.txt FROM logs\iis\ex*.log GROUP BY Pagetype ORDER BY PercentBytes DESC"

42)  Top 20 pages with a specific HTTP return code

logparser "SELECT TOP 20 cs-uri-stem, sc-status, Count(*) AS Total INTO TOP20PagesWith401.txt FROM logs\iis\ex*.log WHERE TO_LOWERCASE(cs-uri-stem) LIKE ‘%.aspx’ and sc-status=401 GROUP BY cs-uri-stem, sc-status ORDER BY Total, cs-uri-stem, sc-status DESC"

43)  Check traffic from IP addresses

logparser "Select c-ip AS Client, Div(Sum(cs-bytes),1024) As IncomingBytes(K), Div(Sum(sc-bytes),1024) As OutgoingBytes(K), MAX(time-taken) as MaxTime, AVG(time-taken) as AvgTime, count(*) as hits INTO errorsperip.txt FROM logs\iis\ex*.log GROUP BY client ORDER BY Hits DESC"

44)  Check errors by IP address

logparser file:errorbyip.sql

Where errorbyip.sql contains the following:

Select
  c-ip AS Client,
  SUM(c400) AS 400s,
  sum(c401) AS 401s,
  SUM(c403) AS 403s,
  SUM(c404) AS 404s,
  SUM(c500) AS 500s,
  SUM(c501) AS 501s,
  SUM(c502) AS 502s,
  SUM(c503) AS 503s,
  SUM(c504) AS 504s,
  SUM(c505) AS 505s
USING
  CASE sc-status WHEN 400 THEN 1 ELSE 0 END AS c400,
  CASE sc-status WHEN 401 THEN 1 ELSE 0 END AS c401,
  CASE sc-status WHEN 403 THEN 1 ELSE 0 END AS c403,
  CASE sc-status WHEN 404 THEN 1 ELSE 0 END AS c404,
  CASE sc-status WHEN 500 THEN 1 ELSE 0 END AS c500,
  CASE sc-status WHEN 501 THEN 1 ELSE 0 END AS c501,
  CASE sc-status WHEN 502 THEN 1 ELSE 0 END AS c502,
  CASE sc-status WHEN 503 THEN 1 ELSE 0 END AS c503,
  CASE sc-status WHEN 504 THEN 1 ELSE 0 END AS c504,
  CASE sc-status WHEN 505 THEN 1 ELSE 0 END AS c505
INTO
  IPNumberFileName.txt
FROM
    logs\iis\ex*.log
WHERE
    c-ip='<IP address goes here>’
GROUP BY
    client

45)  Find broken links

logparser "SELECT DISTINCT cs(Referer) as Referer, cs-uri-stem as Url INTO ReferBrokenLinks.txt FROM logs\iis\ex*.log WHERE cs(Referer) IS NOT NULL AND sc-status=404 AND (sc-substatus IS NULL OR sc-substatus=0)"

46)  Top 10 pages with most hits

logparser "Select TOP 10 STRCAT(EXTRACT_PATH(cs-uri-stem),’/’) AS RequestPath, EXTRACT_FILENAME(cs-uri-stem) AS RequestedFile, COUNT(*) AS TotalHits, Max(time-taken) AS MaxTime, AVG(time-taken) AS AvgTime, AVG(sc-bytes) AS AvgBytesSent INTO Top10Urls.txt FROM logs\iis\ex*.log GROUP BY cs-uri-stem ORDER BY TotalHits DESC"

47)  Unique users per browser type (requires two queries)

logparser "SELECT DISTINCT cs-username, cs(user-agent) INTO UserAgentsUniqueUsers1.csv FROM logs\iis\ex*.log WHERE cs-username <> NULL GROUP BY cs-username, cs(user-agent)"

logparser "SELECT cs(user-agent), count(cs-username) as UniqueUsersPerAgent, TO_INT(MUL(PROPCOUNT(*), 100)) AS Percentage INTO UniqueUsersPerAgent.txt FROM UserAgentsUniqueUsers1.csv GROUP BY  cs(user-agent) ORDER BY UniqueUsersPerAgent DESC"

48)  Bytes sent per file extension

logparser "SELECT EXTRACT_EXTENSION( cs-uri-stem ) AS Extension, MUL(PROPSUM(sc-bytes),100.0) AS PercentageOfBytes, Div(Sum(sc-bytes),1024) as AmountOfMbBytes INTO BytesPerExtension.txt FROM logs\iis\ex*.log GROUP BY Extension ORDER BY PercentageOfBytes DESC"

49)  Domains referring traffic to your site

logparser "SELECT EXTRACT_TOKEN(cs(Referer), 2, ‘/’) AS Domain, COUNT(*) AS [Requests] INTO ReferringDomains.txt FROM  logs\iis\ex*.log GROUP BY Domain ORDER BY Requests DESC"

50)  OS types (requires two queries)

logparser "SELECT DISTINCT c-ip, cs(user-agent) INTO UserAgentsUniqueUsers.csv FROM logs\iis\ex*.log WHERE c-ip <> NULL GROUP BY c-ip, cs(user-agent)"

logparser file:getos.sql

Where getos.sql contains the following:

SELECT
  SUM (c70) AS Win7,
  SUM (c60) AS Vista,
  SUM (c52) AS Win2003,
  SUM (c51) AS WinXP,
  SUM (C50) AS Win2000,
  SUM (W98) AS Win98,
  SUM (W95) AS Win95,
  SUM (W9x) AS Win9x,
  SUM (NT4) AS WinNT4,
  SUM (OSX) AS OS-X,
  SUM (Mac) AS Mac-,
  SUM (PPC) AS Mac-PPC,
  SUM (Lnx) AS Linux
USING
  CASE strcnt(cs(User-Agent),’Windows+NT+6.1′) WHEN 1 THEN 1 ELSE 0 END AS C70,
  CASE strcnt(cs(User-Agent),’Windows+NT+6.0′) WHEN 1 THEN 1 ELSE 0 END AS C60,
  CASE strcnt(cs(User-Agent),’Windows+NT+5.2′) WHEN 1 THEN 1 ELSE 0 END AS C52,
  CASE strcnt(cs(User-Agent),’Windows+NT+5.1′) WHEN 1 THEN 1 ELSE 0 END AS C51,
  CASE strcnt(cs(User-Agent),’Windows+NT+5.0′) WHEN 1 THEN 1 ELSE 0 END AS C50,
  CASE strcnt(cs(User-Agent),’Win98′) WHEN 1 THEN 1 ELSE 0 END AS W98,
  CASE strcnt(cs(User-Agent),’Win95′) WHEN 1 THEN 1 ELSE 0 END AS W95,
  CASE strcnt(cs(User-Agent),’Win+9x+4.90′) WHEN 1 THEN 1 ELSE 0 END AS W9x,
  CASE strcnt(cs(User-Agent),’Winnt4.0′) WHEN 1 THEN 1 ELSE 0 END AS NT4,
  CASE strcnt(cs(User-Agent),’OS+X’) WHEN 1 THEN 1 ELSE 0 END AS OSX,
  CASE strcnt(cs(User-Agent),’Mac’) WHEN 1 THEN 1 ELSE 0 END AS Mac,
  CASE strcnt(cs(User-Agent),’PPC’) WHEN 1 THEN 1 ELSE 0 END AS PPC,
  CASE strcnt(cs(User-Agent),’Linux’) WHEN 1 THEN 1 ELSE 0 END AS Lnx
INTO
  GetOSUsed.txt
FROM
  UserAgentsUniqueUsers.csv

51)  Get timeout errors from the server Event Log.  Display results in a datagrid.

logparser "select * from \\servername\application where message like ‘%timeout expired%’" -i:EVT -o:datagrid

52)  Get exceptions from the server Event (Application) Log

logparser "select timegenerated, eventtypename, eventcategoryname, message into webserverlog.csv from \\servername\application where message like ‘%myapplication%exception%’" -i:EVT

Links

Check out the links below to find more in-depth discussion of Log Parser, as well as even more examples of its usage.

    Silverlight and a technology’s shelf-life – revisiting MIX08

    In the last few years, it seems like more and more Microsoft uses a strategy of "throw it against the wall and see what sticks".  It is likely more perception than reality, but it seems like Microsoft introduces many technologies to great hoopla, only to see them fade to oblivion in a year or two.  Sometimes the technologies are absorbed into other products; other times they just disappear.  To be fair, they’re not the only one to use this strategy… Google Wave, anyone?  It certainly doesn’t make the jobs of application developers any easier, though.

    Now, after growing whispers about HTML5 and what it means for Microsoft’s web strategy, we have last week’s did-they-or-didn’t-they-marginalize-Silverlight controversy at the 2010 PDC (Microsoft: Our strategy with Silverlight has shifted).

    For what it’s worth, I don’t think Silverlight is going away.  In fact, I believe that Microsoft is basically telling the truth in their follow-up statements (PDC and Silverlight).  Silverlight will be the primary development platform for Window Phone 7, and it will be useful for targeted line-of-business applications and web-based media delivery.  It will NOT be the run-everywhere cross-platform tool that was originally promised.  Things have changed, and it appears that HTML5 wins that battle.  It is unclear what this means for continued non-Windows-based Silverlight development (such as new/updated versions for the Mac, an Android version, and so on).

    The Silverlight conversation aside, I thought it would be interesting to revisit the notes that I took at the 2008 MIX conference, and see how other products have changed in the last couple years.  (Note that the first version of Silverlight was introduced at MIX 2007.  I attended that conference as well, but seem to have misplaced my notes.)

    SQL Server Data Services (SSDS) – This was Microsoft’s first attempt at a cloud-based storage service implementation.  Notice that at this time, the name “Azure” was still six months away from being announced.  In the notes I took during the session I questioned how SSDS compared to Amazon’s S3 storage service… and in the post-session Q&A it was revealed that the two services were in fact very similar.  However, Microsoft indicated that “much additional SSDS functionality is expected to be rolled out in the near term (every 6-8 weeks)”.  SSDS was to use standard REST operations (GET, PUT, POST, and DELETE) to manage the data stored in the service, and LINQ for more complicated queries.

    SSDS did eventually come to market as Microsoft SQL Azure, but its released form is radically different than the produce described and demonstrated at MIX08.  Rather than blob storage manipulated via a REST-based interface, SQL Azure is in many ways full SQL Server in the cloud.  Developers can use their existing SQL server tools and skills to work with SQL Azure much as they would with an on-premise SQL Server installation.

    ASP.NET MVC – The first version of this new web programming framework was released at MIX08.  I attended a session given by Scott Hanselman which demonstrated the new framework and pointed out its strengths and areas where it fell short.  I’ve not done much with MVC personally, so I can’t say much about what has changed since that time.  What should be clear, however, is that ASP.NET MVC is a product that “stuck”… it has clearly improved, is respected by the Microsoft development community, and has been made an out-of-the-box part of Visual Studio.

    The Microsoft AJAX Library and Control Toolkit – I attended a session about these technologies, which had already been available for some time (originally code-named Atlas, the first incarnation debuted in 2005).  Some of the tips and techniques shown in the session included methods of controlling Back button functionality, preserving page history, bookmarking, and templating.  At the time, this was Microsoft’s entry into the crowded field of Javascript toolkits.  It was also before jQuery had fully emerged on the scene. 

    Fast forward to late 2010, and what do we find?  jQuery (an open-source product) is now fully supported by Microsoft, and Microsoft is actively contributing to the jQuery product.  The AJAX Library has been combined with the AJAX Control Toolkit and refocused primarily on the service-side aspects of AJAX.  jQuery is now Microsoft’s primary technology for building client-side AJAX applications.  Interestingly enough, all of these changes were announced at MIX10, and described in detail here.

    So what does the scorecard look like?  One new product (SSDS) has made it to market, but in a radically different form.  Another new product (ASP.NET MVC) is a success.  One somewhat established product (AJAX Library and Control Toolkit) has been completely superseded by another technology… one that did not even originate at Microsoft.  All in all, this is about what I expected to find.

    Obviously, this is very small sample size, so it wouldn’t be appropriate to draw too many conclusions from it.  However, it’s not too hard to come up with other products that fit similar profiles.  LINQ to SQL has been surpassed by Entity Framework.  Entity Framework itself was widely disparaged upon its initial release, underwent major changes from version 1 to version 2, and now seems to be more successful.  Project Astoria became ADO.NET Data Services, which became WCF Data Services, which is one of what seems like about 20 different data access technologies offered by Microsoft. 

    It should be clear that very few development technologies remain static for very long, and with newer technologies it is difficult to predict even which will survive from one year to the next.

    These examples underscore the challenges that application developers face as they strive to keep their skills current, while at the same time not wasting precious resources on dead-end technologies.

    Follow

    Get every new post delivered to your Inbox.