Kaleidoscope

Lets start taking at look at some of the NFRs and lets start with Auditing. In some industries, auditing is a very important activity. When we want to audit operations we are looking for a few key things -

1. Who did what action and when did they do it?
2. What effect did the above action have – or what changed?

In some instances, we are concerned about the various actions. We are interested in auditing who looked at some records, who printed some reports, who accessed which part of the application and so forth. This essentially, is a track of actions that a user performs. In a very simple manner, if every user click/input is treated as an action – we are essentially interested in every thing that the user does. Simon has a series of posts regarding auditing which looks at it from the perspective of an Entity-Framework implementation.

But sometimes, the user makes some changes – edits data, deletes some information, approves some items. In such cases, we are also interested to know WHAT data changed. What was the oldisalue and what is the new value that is was changed to. This can get quite messy if an action updates many tables and does a lot of calculations. If we import data and expose the same via services, it adds on to the woes of auditing!!!

ASP.NET MVC has added on a nifty feature called action filters. These are attributes that we add to a method which then calls  the OnActionExecuting (before executing the specified action method) and OnActionExecuted (after the execution). With this we can write a generic audit process that specifies – User X did Action Y at time T and to audit any action, we just need to add the attribute to it. If we generate an ‘action-guid’, we can now start tracking the various actions that users across our system are performing in our application. This is nice, but is not the total solution.

While the ASP.NET MVC helps in tracking all actions that users perform (or actions that external systems perform – the Controller methods are exposed to the outside world. So any action filter on these methods will also be applicable to external services), it does not tell us what has changed.

This is where the SQL 2008 change-data-capture or CDC comes into play. The CDC can track all the old values and the new values and it does so in the background for us. In order for this to be useful, we need to somehow pass on the ‘action-guid’ so that we can related all the data changed to the operation that was performed by the user. This can now give us comprehensive reporting on who did what action and as a result of that action, what data was changed. That is a good set of information from an auditing perspective.

Note also that SQL 2008 introduces an AUDIT object which we can use to track who did what – who executed a proc, who selected data from a table or who updated/deleted records from a table. If you are using the EntityFramework, you can add-on auditing to the object context like what Simon describes. So depending on what set of technologies we are working on, we can use these features to make auditing easier and part of the underlying engine in our applications with minimal effort.

As we start out to build systems, we – as software engineers – are expected to take care of ‘engineering’ the system. When we go out to buy a car – do we get asked for our requirements on securing the car? do we get asked on what load the car needs to take and what all we plan to do with it? When we move into a house are we asked what load the roof should take and do we have any plans of partying on the roof? There are some things that are engineered. When we build a bridge, it can take the weight of a flight landing on it – but it might get washed away in a major hurricane. So one of the concepts that we learn in engineering is to add on a ‘weighted factor’. Any calculations that involves lives – multiply by a factor of 18-28.  If we want to structure to face rough weather – multiply it by 50 – and so on. These factors have evolved over time and have become common industry practices. Unfortunately in software, we don’t have a common engineering practices yet. This allows one vendor to cut costs by cutting corners over ‘un-asked’ for functionality. Its like a car manufacturer who says – hey you asked for a car, but you did not specify that it needs to have a door.

This brings us into the realm of NFRs or non-functional requirements. Some call them frameworks, patters and practices, design patterns and the like. We sometimes over-design our systems and many a time under-design the system. Lets start taking a look at some of the common NFRs and see how we can design and build them. Over a period of time, lets assimilate them all with the hope of having an ‘acceptable’ framework for building solutions. In the first run, let us also limit this to one type of solution – a new project for a customer, designed and developed from scratch on the ASP.NET MVC2 framework using SQL2008R2 at the back-end. For this solutions lets start by taking a look at some common internal aspects like security, navigation, auditing, error handling, search, data access, logging etc.

Keeping your Windows regularly updated is important. But it can also be an administrators nightmare. One of the options that Microsoft has, is to install a local WSUS (Windows Server Update Services). This done, it essentially becomes your ‘local’ Microsoft server from where all updates can be served. Saves on bandwidth – where instead of all the other machines downloading from the net, they can get the updates from this server instead.

Well it does not stop there. WSUS also allows you to have a much greater control. To being with, you need to update your clients, to use your local WSUS server and not the regular Windows Update settings. You can, now, on the WSUS server, decide which of the updates to authorize and which ones not to. You can also group your various machines into groups and choose which updates to apply to which group. And your can also have exclude lists where if one of the machines does not need to be updated with a patch, you can exclude it from that list. All the administration on the WSUS server can also be done using the WSUS SDK.

In case, you want an even better control, like one of our customers did, what you can do is to use WMI to configure which updates need to be ‘activated’ on which client. The client is setup to use the local WSUS server. In this way we have a very granular control on which machine needs to be patched with what software. Up until now, they were attempting to get the patches and ‘install’ it themselves. Not only is this messy, but when installing patches through msi or exe files on a remote machine, they will start hitting up onto the Windows UAC in Vista and Windows 7. By leveraging the existing Windows update service that’s already there, the entire process continues to remain clean and easy.

One of the nice backgrounds to the Windows Update is the concept of BITs. A while ago, one of my fellow RDs, Chris Auld pointed me to BITs. BITS – or Background Intelligent Transfer System – is a great piece of technology that copies over files (like windows update patches). The great thing about BITS is that it ‘JUST WORKS’ – across firewalls, across setups and across most environments. In fact even for our local file copies, we have started using BITS. It does the copy in the background, it can ‘downgrade’ its own priority when there is a lot of activity on the machine, it can resume from where it left off for large file downloads – making it soo nice. You can learn more about BITS here.

One of the things that I wanted to do was share my experiences of working on various projects. In most instances, problems come my way, and I start working on them. At that time, I have only a notion of the problem and a very vague idea of the solution framework. Many a time, lots of attempts don’t work and we finally end up with a satisfactory solution. During all this time, I was uncomfortable blogging about it – as I myself was not clear about the thoughts.

Then I hit upon an idea. I like the way ScottGu aggregates his blogs into links which essentially give a storyboard to the whole process. So as part of case studies, I shall attempt to do the same. I shall give a unique case study tag to each problem that I see and am working on. As it progresses, I shall continue to tag it with the same case study so that over a period of time, it gradually progresses into a storyboard. Rather than categorize case studies, I’ll just tag them, making my life easier

One of the discussions that I was having with a kid recently was about taking steps to save the world. The kid in question was arguing that she wanted to use less paper in order to reduce the felling of trees – a very noble thought indeed. But does it really matter at all?

The crux of the matter is that the paper manufacturers don’t yet know that we want to reduce the felling of trees. If you track the life-cycle of the tree from the time it blossomed to the time it became a robust tree (all of 60 odd years or so), to the time it was cut down mercilessly and shipped to mills where it was butchered down (maybe another year or so), to the time it was pulped into paper and stacked in paper mills, from where it was cargo-ed out to multiple warehouses which stock pile the paper based on their estimate of its use in the coming months (years), to the time someone buys it and uses it…. its a long cycle. So if we don’t buy paper, we are essentially leaving some of it in the stores, which then make up innovative plans to get rid of it (discounts, corporate offers..). If the store sees a ‘CONSISTENT’ reduction in requirements, only then does it start stocking less from the warehouses. Again as the warehouses see a ‘CONSISTENT’ decrease in requests does it start alerting the mills, who then trigger the alert up the chain. As and when this feed reaches the woodcutters, ONLY then would there be a reduction of cutting trees.

Now the issue is the definition of ‘CONSISTENT’. After a continuous growth in requirement of paper over a sustained period, any dip will be thought of as an aberration. Who would want to start reducing the output and stare at a point where they don’t have stock to match customer requests??? Its the ‘fear’ or more appropriately the lack of real-time market touch points that create a system that’s lethargic and leads to waste.

This in many ways reflect what we do in software as well. If there are features that someone wants or if there are feedback, when a company takes ages to roll out the updates/upgrades, then the whole process becomes redundant. One of the reasons for the growth in open-source/shared-source projects is that people at the ‘fag-end of the usage cycle’ can update the project. It like you and me, having a say in which trees and how many to cut! Cutting down this cycle of action and reflection is very critical for anything to be more effective.

I didn’t have the heart to disappoint the kid and whole-hearted supported her efforts in reducing paper in her own small way. While it might not make an immediate difference, the thought might just one day count!!

The more I look at the iPad details, the more I get the feeling that it is a lot of what I want. The hardware does look funny (can’t imagine someone holding up the iPad to talk over phone, typing might not be as easy when I cover part of the screen with my pudgy fingers). But it does one great thing. As I was talking to a pal of mine, one of the first things that I want my computers to do, it to be ALWAYS ON. Or at the least I should be able to turn it on like I do my TV – with the click of a button. The days of a computer ‘booting’ up are ohh so archaic. In fact this was one of the feedbacks that I gave the Windows team of what I would dream of for the next version of Windows. Does it really make sense to ‘boot’ on my laptop just to order a pizza, check mails, check scores of a cricket match or watch a movie!!!

Now I got another thought. Maybe – just maybe, Microsoft can consider doing all this as part of its Live Platform. IF (and thats a big if!), Microsoft actually turns around Live as the default OS that a computer/mobile device has (the so called competitor to Chrome/iPhoneOS/Android and its own Phone Series 7). It can actually ship all its services (Live Browser, Live Mail, Live Writer, Live Movie maker and watcher, Live Photo Gallery, Live Mesh, Live Mobile Sync…. and so on) as a set of apps that can be used EVEN before ‘booting’ into Windows. And if that goes cross platform – oh… I would soo love to Switch on Live and then, only much later ‘boot’ into Windows. Given that Microsoft is coming up with the web versions of Office – a ‘always ON Live platform in ROM would be so nice.

Given that Live is anyways free, this can become the defacto installation on most machines. Windows can then be a BOOTFROMVHD only option – an Microsoft can just ship Windows VHD files that need activation.

So what’s Pranab Mukherjee doing on my Featured Video?  Well, its our step towards starting to build out content on topics outside of technology. In cojunction with some CAs whose forte is accounting and taxes, we are working at creating a set of Financial content. A few things that we tried out with this is the voice recording. Whenever someone does not know the topic they are taking about, the voice does seem ‘disconnected’. But somewhere out there – there’s an acceptable limit. We are seriously trying to hit that ‘acceptable’ limit. The main thought process is also that if we want the SME – the subject matter expert – to actually sit down and voice-over the content, it takes a lot of his/her time. We need a way to ‘outsource’ the voice-over as long as its within ‘acceptable’ limits and then get the SME to lend further credence to it when they are available. Give me feedback on if this voice is ‘acceptable’ (or at least ‘palatable’) or not.

On a separate note, I was looking for a Notes Editor. Yep – for most notes that I take Evernote has become a good companion (Actually, I like the Mac version of Evernote more than the Windows version). But when we want to get down to the serious job of writing, I thought I need something better. So what featrues did I really need?

To begin with, like when we are writing a book, I need to be able to have a near full screen view of my editor. I literally ‘bang away’ on the keyboard a the thought flows. I have not found a way to move to a ‘full-screen’ writing mode in Evernote. Secondly, I really dont want to organize my notes on a time-scale. I know evernote gives me the option to order my list by one of Title/Size etc. But somehow that’s not really the same. Thirdly, I could not find a way to link notes with each other. Like a TOC from where I can link to one of the other notes. Microsoft OneNote give me that! But I did not seem to find that in Evernote. This was a bit of a killer for me – As we progress in writing a book or a series of articles, we would want to first put up a TOC and then link to the content as we get them done. That way the TOC page is the base page.

Exploring Jarte : That’s when I hit upon Jarte. Not as great as evernote. And does not do the most important – online sync – for me. But it seems to be more idealy to sit down and write a book or a series of articles. (Also note that my SME’s are not always tech. They could be Chartered Accountants, Health care specilists, K-12 teachers). Jarte is essentially a wrapper around WordPad. It gives me a near full screen view to ‘bang away’ my notes or thoughts. Is light and portable (For the records, Evernote has a portable edition too). I can keep multiple tabs open at any point in time, thus giving me the ability to have multiple articles/thoughts/chapters in limbo as I work on each of them. It can organize the content into Folders and Files (which essentially is a peek into Windows Explorer itself – so it just book marks some of the folders and files from explorer into Jarte). It has ‘favorite files and folders’ and keeps the last set open. It does not need a save as it can be configured for ‘auto-save’ and also for ‘auto-backup’ if and when u mess up some of your notes. You can add-on HyperLinks to either the web or to local files (which was great for me to have a TOC doc and then have all the others as independent docs). It supports rtf and doc files. I just want to use the rtf formats for now.

One of the nice addition is that it has an in-built screen capture. Sooo useful to take screen shots and add onto your notes. If we take multiple screen-shots, it organizes the same into a clip-list manager so that I can re-use the same screen-shot anytime I want. What it lacks is a online sync – but hey that’s what DropBox is there to provide In a sense this gives me the best of both world – I can keep things locally along with a backup as long as the article is not too serious, and then post it up online when I’m done.

Evernote will be my constant companion when I want to take notes, work on projects and so forth. But as of now, I feel more comfortable in Jarte to ‘bang away’ articles or posts. I am hoping that this will help me in my endeavour to write a book this year.

After some struggle, I finally got VHD Boot to work. Not that it was tough or that it is a struggle – far from it. But my scenario and what I was trying to do was a bit different (isn’t it always ). I wanted to setup my MacBook to dual boot between Snow Leopard (10.6.2) – which I would want to use for operations (Mail, Contacts, Calendar, Sync, Music, Videos, Pic etc) and Windows 7 – which is my core work area (Visual Studio + SQL) and which I mostly live on. Given that I mostly used Windows, I constantly ended messing up my machine with various installations (GoMeetings, ISOMount, Firefox and Gears, Chrome, RTM, LiveWriter, Live Movie Maker to name just a few…). I really really didn’t want to mess up my Windows environment too much. And that is where VHDBoot came in.

With VHDBoot, we can have a base version of Windows 7 which ends up being very clean, and then have multiple virtual machines installed into VHD Files. But instead of loading these in a virtual environment like VirtualBox or even Hyper-V, we can actually boot directly into these VHD Files. I got started off with Scott Hanselman’s excellent blog about how to go about setting up a VHDBoot. But what I wanted to do was to actually keep the VHD Files on my external 320GB/7200RPM hard-disk. As Scott mentioned, this is not easily done – but from all I have heard this seems to be in the realms of the possible. This would have made my environment so portable.

But as I thought about it, I realized a few things. Firstly, once I VHDBoot into the machine, it sets-up all its drivers for the base machine (in my case the MacBookPro). So it’s not as if I can really use this elsewhere. (If there are set of guys all on a Toshiba or Dell, maybe they can all share the VHD, but hey – that’s not what I was actually looking for). Secondly, once I VHD Boot, I’m guessing (still need to try this out) that I wont be able to use if from within VirtualBox. This would be ‘cos the VHDBoot loads all the core drivers onto the machine. If we boot inside of VirtualBox (or VirtualPC  for that matter), they would install virtual drivers to emulate the physical drivers and this (I’m guessing) would badly mess up that environment.

But the HUGE benefit I have is that the performance within the VHDBoot environment is awesome. It is a Windows Server 2008R2 (64-bit) which has access to ALL of the 4GB RAM. And of course it also has access to ALL of the drivers so I have no concern in setting it up. The only downsides (in my environment) that I see – I have a installation of Windows 7 which is technically useless to me. It will be nice if I can replace this with the Hyper-V Server R2 or a core server – that way the unused part of my installation is minimized. The only other gripe is that I will have the same disk on which my base OSes (SnowLeopard and Win7) are running and in which I am maintaining my VHDs. It might have been nicer to have them on separate disks which might have given me a better performance. (That benefit is offset by the fact that internal disks are faster than through a USB and of the fact that one of only 2 USB ports on my mac is not taken up just to boot into an environment).

In all – this is something that I’m falling in love with. Now, if I can get VHDBoot to run off a Windows XP machine as described in this post by Ramnish won’t that be sweet.

One of the big expectations from each release of any major operating system like Windows, Mac OSX, Linux Desktop, Google Chrome OS(!), is that it simplifies our life and makes things easier. I don’t think that we really want to be learning how to use a computer all over again. With Vista, Microsoft introduced a host of features to make the PC simplified and yet secure. The issue with Vista was that it wasn’t all that stable. Windows 7 is in many ways what Vista should have been in the first place.

Windows 7 continues the tradition of a clean uncluttered desktop. It enhances the taskbar and the sidebar and attempts to create more real estate for the user. But the most significant thing that we notice about Windows 7 is all under the hood and not easily visible. A 12 sec boot time, a fast shutdown, quick launch of most applications, a fast response to user actions and compatibility with most applications (which wasn’t the case with Vista) makes Windows 7 a pleasure to get started with.

Windows 7 streamlines and simplifies the tasks you do most often. Its enhanced Start menu and Windows Taskbar put the files and programs you use most within easy reach, so they don’t clutter your desktop. And the desktop’s new streamlined interface puts you in control so you can get where you want to go faster. To end users, the Windows Live Essentials provides a package of services that have become enmeshed into our lives including – Live Mail, Messenger, Movie Maker, Photo Gallery, Writer and Family Safety.

With an improved search engine and Windows Explorer in Windows 7, you don’t have to wade through multiple results to get the files or programs you’re looking for. No matter where files are located on your PC or on your network, finding them is a snap. Windows 7 extends the ease of use of the desktop onto the web. Browsing the Web is simpler too, thanks to new features in Internet Explorer 8 that puts the information you’re looking for right at your fingertips. Better device management features make being mobile with Windows 7 much simpler.

Like we mentioned before, superior performance, reliability, and security enhancements make your PC work the way you want—that is, without fuss and with fewer problems. Windows 7 is designed to start, shut down, and resume from standby faster. Windows 7 also helps extend the battery life of your portable PC (note : ensure that you have the latest drivers for your hardware installed)

One of key focus areas in any Windows release is security. Vista brought in the UAC – user account control that made the OS a fort. But the experience was not very smooth for the user and it also led to a lot of application incompatibilities. In Windows 7, Microsoft has continued its endorsement and trust in the UAC based security, but has made it a much simpler experience for the users. With BitLocker (for data encryption) and Parental Control (for secure web browsing), Windows 7 continues to focus on making the OS more secure along with a simpler user experience for the user.

One of the other ‘under-the-hood’ features is the self healing process built into the Windows OS. Keeping your PC running smoothly—and finding out what to do when there’s a problem—is much simpler now because you’ll be able to resolve most issues from one central location. You’ll recover from problems faster with self-healing diagnostic features and troubleshooters. In fact, Windows 7 will often fix issues without any impact to you.

One of the things that people love about Microsoft (and there are various reasons for which they don’t too) is its commitment to users on their platform. Windows 7 continues this tradition in keeping the user in mind. Need to run older business and productivity programs that require Windows XP? Now you can, with the new Windows XP Mode at no cost at all. Plus, you can make your PC truly your own with more options than ever for you to customize and personalize your PC to match your personality.

But Windows 7 also builds new bridges by delivers enhancements that will help you do exciting new things with your PC. Windows is gradually becoming the media hub of a home. Windows Media Center and Windows Media Player help you enjoy your media in new ways and places. Are your family photos located on a different PC in your home? No problem. What if you’re away on business and want to listen to a playlist that’s on your home PC? Believe it or not, you can do that too. Windows 7 gives you a seamless experience for your corporate and personal data irrespective of where you are.

In line with the industry directions, Windows 7 also lays stress to a much improved graphics experience. Improved games, High Definition graphics support and a much better 3-D rendering engine makes Windows 7 a touted platform for games and game developers.

Windows 7 also makes it easier to move beyond the traditional mouse and keyboard and interact with your PC using your fingers and multi-touch gestures to control what happens on your screen. This now makes it practical to have a computer in the kitchen or the car where a keyboard and mouse input is impractical. This also lends itself to be used in locations like a mart or a construction site.

Windows 7 will be available in six different editions, but only Home Premium and Professional will be widely available at retail. With the exception of Windows 7 Starter, all editions will support both 32-bit (x86) and 64-bit (x86-64) processor architectures. With most of the hardware now running on Intel DualCore, the hardware is inherently 64-bit. But remember that you will see significant benefit of the 64-bit platform only on high end uses of the systems and with a RAM of around 8 GB.

I was caught up between upgrading to Snow Leopard or (!) putting just Windows 7 on my Mac. Most of my work/life is in Windows. While emails, contacts, calendar etc works across both OSes quite well, I realized that moving across OSes is wasting a lot of time. Along with that was the option of working in a Virtual Environment using VirtualBox – which again works across different OSes. So after a lot of thought, this is how I decided to go ahead with my setup…

1. On my MacBook (White, non-unibody, Core2Duo, 4GBRAM, 80GB HDD (!!)), I created 3 partitions – MacCore, WinCore and Data. I did a fresh installation of Snow Leopard onto MacCore and a fresh copy of  Windows 7 to WinCore (note – I did not use BootCamp – felt for some reason that BootCamp might not allow me 3 partitions). Then I installed the BootCamp (Vista) Drivers onto Windows 7 thus giving me most of the drivers for Mac (Apple has promised to release Drivers for Win7 by early Jan).

2. Now that I have setup the machine, I have a BIG problem . How do I go about using Virtual Environments. I have 2 options – Use Virtual Box and have the HDD on an external driver. I went about getting a 320GB 7200 RPM external drive for this. The second and ever nicer option is to do VHD Boot – but I have a fight getting it to work with my setup. To get the VHD Boot to work, I followed the excellent steps from Scott Hanselman’s blog. But like he said – we still can’t boot from an external USB drive. If I can get that to work – I will be mostly done. I am on Windows 7 RC and will look for a solution before I move to the final RTM build

Any suggestions – Should I move to a 64-bit Window 7 or a 32-bit one? I am only concerned that all the drivers might not work on the 64-bit. Also I don’t see a benefit as most apps will still be 32-bit!.