Hello, everybody, I have released the first version of a rudimentary managed wrapper around Transactional NTFS to Code Gallery on MSDN: http://code.msdn.microsoft.com/txfmanaged.
The wrapper in its current form is very basic but should give developers a good idea on how simple it is to use Transactional NTFS from managed code. Feel free to go grab it, play around with it (there are some sample screencasts using the wrapper on the Developer Meet Server Show), and feel free to use it any way you wish.
Enjoy!
James Conard announces over on his blog that our team has just released the Windows Server 2008 Developer Training Kit for Beta 3. Mosey on over and check it out. You can find resources on everything from WCF, to Cardspace, to WF, to Windows Eventing, to PowerShell, to IIS 7, to Transactional NTFS.
So if you are a developer interested in checking out Windows Server 2008, make sure to grab it.
Well, it's official. Beta 3 of Windows Server "Longhorn" has launched. With it, I finally get to release some of the work I've been working on lately.
If you are a developer, I encourage to take a look at "The Top Ways To Light Up Your Apps On Windows Server 'Longhorn'" that was just launched today. It outlines key ways that you, the developer, can leverage all the new and improved enhancements to the Server platform to really make your applications shine in a whole new light.
I encourage any of you with access to Beta 3 to try it out. Windows Server "Longhorn" is quickly shaping up to be the best Server release from Microsoft ever.
I recently posted the second screencast in my new "Developer, Meet Server" screencast series. This latest screencast covers using Transactional NTFS in a service-oriented environment via WCF.
Enjoy!
Andrew recently watched my screencasts and had some great questions that came up around how Transactional NTFS behaves with the file system. So, I figured I would answer them here. So, if you want to see the context around his questions, check out his post.
Here's the paragraph with the questions:
I wonder what happens to files that never get committed - ala a long running transaction. They have to be taking up file space. Jason says not even Windows Explorer knows about them, but they have to exist somewhere. What if they overflow? Is there a way to clean house? If the poop is invisible, does it still stink?
Great observations and questions Andrew! In the case of a long running, open transaction, the files will exist on the disk. Hence, they are certainly taking up space on the disk. If you have 256 MB of free space on your disk, and try to write 257 MB of data to the disk within a transaction, you will get an "out of space" error like you would outside of a transaction.
Is there a way to clean house? Yes, rollback or commit the transaction. On rollback, any changes made within the transaction are rolled back, and then consequently removed form the disk.
But what if my server crashes in the middle of a transaction. Would I have turds floating around the file system? No, you would not. When an NTFS-formatted volume is mounted in Vista or Windows Server "Longhorn", any transactions that are "in-doubt" (the transaction manager is no longer around and we have no clue on the status of the transaction) will be rolled back automatically, effectively removing all the changes from disk. So, in that sense, you would not have any file turds you have to clean up after a system crash (like you would have if you were doing a "temp file -> rename" type of methodology.
Now, as is usual in transaction systems, you have to be _very_ aware and _very_ cautious of long-running transactions as it could cause some long-term problems that you don't want to deal with. So, there certainly are scenarios where Transactional NTFS just is not a good fit. As tends to be the case with all technology, make sure you are using the right tool for the right job :).
Welcome everybody to the inaugural, first ever, grand opening, “opening the doors to the public” celebration day kick-off for the new “Developer, Meet Server” screencast series. I am your sincere, humble (as ever), gracious and wonderful host/chef/tour guide Jason Olson.
In this series of screencasts we will be diving through all the various cool developer-oriented enhancements on the Windows Server "Longhorn" platform that developers can leverage to build robust and powerful applications. Specifically, in this first screencast, we will take a quick peek at Transactional NTFS. In future screencasts, we will go into a more detailed look at Transactional NTFS, so think of this as a quick introduction.
The screencast can be found in one of the main tubes on the Intarw3b known as the ever-lovely Channel 9: http://channel9.msdn.com/Showpost.aspx?postid=289816.
So sit back, grab a cup of coffee/tea/InsertYourFavoriteLocalTastyBeverageHere, and enjoy this screencast.
I recently came across a bunch of stuff done by Daniel Moth, a Developer Evangelist for Microsoft UK. If you are a managed developer and want to see the new stuff you can leverage on Vista, be sure to check out his stuff.
If you want to digest some of these new features in Screencast format, be sure to check out some of these screencasts that Daniel Moth has also done:
Enjoy! And thanks go to Daniel for all of these :).
Rob Atkinson recently blogged some screenshots from a new tool from NeoSmart called EasyBCD. What is EasyBCD? Directly from the horse's mouth: EasyBCD is NeoSmart Technologies' multiple award-winning answer to tweaking the new Windows Vista bootloader. With EasyBCD, almost anything is possible. Setting up and configuring Windows boot entries is simple, and there is no easier way to quickly boot right into Linux, Mac OS X, or BSD straight from the Windows Vista bootloader - on the fly, no expert knowledge needed! EasyBCD is geared for users of all kinds. Whether you just want to add an entry to your old XP partition or want to create a duplicate for testing purposes; if you're interested in debugging the Windows Kernel or hectuple-booting your seven test operating systems, EasyBCD is the key. Automated MBR and BCD backups, boot sector restore kits, support for a dozen+ operating systems, detailed configuration of all boot entries, and award-winning guaranteed technical support is what makes EasyBCD stand out - all for free! Give it a shot!
Howdy all y'all. I'm hoping to get some feedback from all of you. Basically, I'm hoping to get a "reader's braindump" on what development features you are looking forward to most in Vista/Longhorn Server. So, what development features are you looking forward to most in Vista/Longhorn Server? Feel free to just leave a comment (nice and easy-like) :).
For anyone that has had to boot Windows into debug mode to do some digging around (like remote Kernel Debugging), things have changed. In the past, you could open up boot.ini in the root of the system volume, and add a "/debug" flag to the Windows OS record of choice. One of the things you might notice with Vista is that boot.ini is now gone, solid gone. So, with boot.ini gone, how do you boot into debug mode? It is done with a new tool called "bcdedit". In the root of the system volume, you'll notice a hidden system directory "Boot" which is the Boot Configuration Data store. So, to turn on Kernel Debugging with Vista, you can now execute "bcdedit /debug on" (be aware, you need to be running the command prompt under an account with Admin privileges). If you want to see the load of stuff you can do with bcdedit now, just execute "bcdedit /?".
I'll be updating this post over time to reflect new resources that are made available regarding the transactions platform in Windows. Websites: Blogs: Specific blog posts: Media:
If you feel I have left any out, please let me know so I can update my list :).
Jim Johnson has a new post about how to use TxF with the new model in Vista RC1. You will obviously need the RC1 bits to mess around with it.
I'm hoping to expand on this topic in the future to show some code samples of using it not only by itself, but also using it with SQL and the like. I'll also have some "context" posts around that describing the architecture and relationships around TxF so you can better understand the articles.
Until then, go check out Jim's post :).
A friend of mine made a good comment on my last post about TxF: "what the heck does TxF do and why should developers care?" I hope to answer that question in this post. If you've never heard of TxF, read on. If you have and still don't "grok it", read on. If you just happen to love the wonderful language that is expelled from my tranquil appendages as they dance over the keyboard, then by all means, read on as well :).
So, what is TxF? TxF is the "nick name" for a new technology coming with Vista and Longhorn Server: Transactional NTFS. You may have read about or used logging file systems in the past, and you may have used journaling file systems in the past as well. But TxF is a fully-transactional file system, all built on top of NTFS. And when I say "fully-transactional", I mean it supports full ACID transactions on the file system itself.
Why should you care about TxF? You should care about TxF because the transactions that operations are contained in are not only specific to the file system. When working with TxF, there is full support for the Distributed Transaction Coordinator as well. This means that now I can coordinate transactions between the file system and SQL. Heck, this also means that I can coordinate transactions between the file system and Windows Communication Foundation.
Just stop for a second, and think about the scenarios that this opens up to us as developers. Let's look at a Document Management System, for example. If I were to submit a new version of a file into the system, the database may have to store a hash of the file as well as a pointer to the file on the file system. The problem is that we have to now guarantee that the hash of the file in the database in identical to the version of the file on the file system (I'm of course assuming that we aren't just shoving everything into blob fields).
If we update the file on the file system first, there are at least two situations that can occur that we might have to deal with: 1) If someone tries to pull down the file before the database has been updated, then the hash comparison will fail and the file will falsely be identified as corrupt. 2) If the database update fails for any reason at all, we now have to write extra code to roll back the file manually, and in the meantime, users pulling down the file could have it falsely identified as corrupt again.
So, let's change the order. Let's say we update the database first and then the file. Unfortunately, we're not in a better position, only reversed from before. Either way we go, there is more code for us to write if we want a stable system. This is where TxF can come in to make our systems more stable and robust. We can use TxF's ability to coordinate with DTC to manage all of this for us.
First of all, it would be completely Atomic. By using DTC for our database update, and TxF for our file update, they can coordinate with each other to guarantee that either they both succeed or none of them do. That's right, if the database update fails, the file system update will be rolled back, and vice versa. The transaction is also Consistent. At the same time, the file system update is also Isolated. After we update the file and before we commit the transaction, every one outside of the transaction will still see the old version of the file (how cool is that?!?!?). And the system is Durable. If we commit the transaction and the system crashes, when it restarts, the file will be guaranteed to be updated. And all of this is handled by TxF (well, handled by the new Kernel Transaction Manager, to be specific).
That's just the beginning of the story though. Imagine this on a distributed scale. We can use TxF to manage file operations across multiple machines. We can also coordinate TxF with WCF using System.Transactions :D. So, let's say I wrote a Music Download service. As a user, I want to ensure that my credit card is not charged unless I download the file completely. This is a scenario that becomes a lot easier with TxF. If my credit card was not able to be charged, I should be not be able to keep the file. On the other hand, if I was not able to download the file successfully, my credit card should not be charged (disclaimer: there are some obvious security holes to work through in this example :P).
By adding transactional support into the file system, we are also hoping to make your lives easier in the long run. You no longer have to write extra code to self-manage the file system when dealing with distributed systems. By leveraging TxF and System.Transactions, all you basically need to do is code against "The Happy Path." You just have to ensure that if anything fails, all of it is rolled back (which is ultimately handled by the system). There are a couple of sayings I like to keep in mind as a developer: 1) The fastest running code is code that doesn't run at all (I believe I got this one from Rico M.) and 2) Code that isn't written is code that doesn't have to be tested. Easier lives through transactions :).
It's not all glitz and glamour though. In my last post on TxF, I discussed some of the "down side" of TxF as the API exists today. There were two "features" that are not implemented that I didn't mention. Mainly, at this time, we do not have support for transactions on streams, or transactions when dealing with encryption. The latter, I am told, is because of some interesting security challenges that we need to iron out first.
One of the challenges I have ahead of me as a Technical Evangelist is "spreading the word" of TxF. Even before I joined Microsoft, I had only heard about it because of a couple of things: 1) the job posting for the job I interviewed for and 2) A video about TxF made by Channel 9. There really isn't a lot of information out there on TxF. And the information you can find is largely based on Vista Beta 2 or earlier, meaning, it is now quite out of date.
When chatting with Carlos about it, he said (and I quote): "I read a lot of blogs and that's the first I've ever heard of it." I'm pretty sure he's not the only one that would have this same reaction. Time to spread the word, my friends :). I only hope I can get a fraction of you out there as excited about TxF as I am. If I can do this, then hopefully the technology will speak for itself and word of it will spread fairly quickly.
[Usual Disclaimer: my opinions in no way reflect or represent the opinions of my employer]
You'll probably hear me start cheering for TxF after a couple of posts because it is a great technology, but at this point in the ballgame, let's take a step back and take a sobering look at the state of TxF as it exists in post-Beta 2 Vista.
As I've been getting up to speed with TxF, I've realized that there is very little up-to-date information on TxF available (except for an internal presentation that was given at an internal conference). Most likely, any blog posts you'll find on the internet regarding TxF and how to use it are no longer accurate because of the newly revamped API. The extent of up-to-date information even available internally is constrained to a single presentation by the PM and the header files for the various APIs. Of course, it is one of my responsibilities to contribute to the amount of content that is available, so I suppose you could consider it job security if I do my job right :).
TxF itself, is a VERY cool technology. And to put together a managed code example of how to use the new KTM to manage transactions across files simply on the local file system is nearly trivial (you can expect to see a Screencast on Channel 9 from me on just how to do this). However, the truly interesting and compelling scenarios (I believe) don't exist simply with only files on the local file system. To me, the interesting scenarios are when you are managing transactions that span SQL and TxF, or WCF and TxF, or heck, even SQL + WCF + TxF. However, when we get to these scenarios, the "story" breaks down a bit.
If you are a C/C++ developer, you're fine, you're clear, you're out of the woods, you've been cleared for a safe landing, you can use this stuff full-force out of the box. If you are a managed code developer on the other hand, buckle your safety belt folks, we're going in for an emergency landing. To me, this part of the story, frankly, SUCKS (and trust me, we know it sucks). You heard it correctly folks, as of the post-Beta 2 API, there is no longer any Managed API into TxF. If you want to use TxF from C# (and have it work with SQL/WCF via DTC), you will be following the brown-bricked road into P/Invoke and Marshal.QueryInterface land.
The removal of the managed API can partially be attributed to the removal of the implicit transaction model, which brings up another unfortunate side-effect. When the implicit transaction model existed, you could leverage TxF with little to no code changes. You could basically instantiate a TransactionScope, have KTM enter the transactional context (EnterTransactionScope/ExitTransactionScope in kernel32.dll), and then utilize the normal File class APIs. Since the transaction model was implicit, the files would (you got it) _implicitly_ be enrolled in a KTM transaction that is being coordinated with the TransactionScope.
With the implicit model having been ousted now, there are a series of new transacted API calls that exist in kernel32. For example, there is now a CreateFileTransacted, OpenFileTransacted, etc. and they all have almost the exact same interface as their corresponding un-transacted siblings (the only real difference being that the *Transaction API calls accept some additional transaction-oriented parameters). As you can see, it is very much an explicit programming model now. Unfortunately, this means that if you have an existing product, you will have to change any file system calls necessary to leverage the new Transacted APIs. It's not too bad, but it still sucks in my humble opinion. Of course, for any managed code developers out there, it sucks more because instead of using the File class in System.IO, you now have to P/Invoke into kernel32 to get at all the Transacted APIs.
In my opinion, the challenge this presents us as a company is that if we want our managed developers out there to use this, we absolutely CAN'T expect them to use it as it exists today. I think if we tried to do that, adoption would fall flat on its face. My current thoughts are that we can perhaps release a publicly-available community wrapper (maybe on GotDotNet, CodePlex, or something else) that people can use to easily interact with TxF with very little effort. I also believe that for even that to be successful, it can't be a "totally new" API. We should leverage the existing look and feel of APIs like System.Transactions and System.IO.File to provide the TxF functionality. I'd imagine this is one effort that I'm going to try to champion and to get out there for all of you. Heck, if not for you, I'll just do it selfishly just for me since I don't want to be constantly context-switching down into P/Invoke, COM madness when writing my screencasts, demos, and other TxF code.
Don't get me wrong, there has been a lot of cool work done. And all of the reasons I've heard on why the implicit model was "broken" (or at least that implementation of an implicit model) make absolute sense. It is just another example of what you have to go through when you have to worry about backwards compatibility, app-compat, etc. Yes, in this case, it has given us a bit more work to do. But I can live with that :).
Why am I making this post? Am I a pot-stirrer? I don't believe so. I simply want to get the truth out there so you all know that there are some warts we need to remove on the product. More importantly, we are recognizing these warts and are actively engaged at trying to remedy them. I truly hope this does not dissuade you from trying it out as it is certainly very cool stuff and shows a lot of potential moving forward (which I hope you'll see as I dive into it with all of you :D).
Link to Microsoft gets good reception at Black Hat | CNET News.com I found this rather interesting. Of course, it was a small sampling of attendees and there were some others that thought it was just a huge sales pitch. Nonetheless, I'm please to see Microsoft going out and leveraging the skills of some of the best hackers out there to make Vista a more secure operating system.
Last night I upgraded to one of the newest internal builds of Windows Vista and, wow, it's definitely gone through some more polish. Problems I had on previous builds are gone (for the most part (not that there aren't some other new problems around having some questionable drivers)) and here I am actually posting from Vista. Sexy, sexy, sexy. It is shaping up to be one sexy product, that's for sure. There have definitely been quite the number of performances made between the first Beta 2 and the build I'm currently running on (it's fast, baby). (I honestly wonder what Scoble would think if he was able to see this latest build).
That reminds me, did y'all catch the news of what Mark Russinovich is working on now that he's a Microsoftie as well? Quoted from Mark himself: "For the near future I’ll be working with the client performance team analyzing and addressing Vista performance issues." That's pretty exciting to me. And with his skills, I'm sure it won't take long until he starts making an impact in that area.
I guess in one way, it's the small things that make me happy. The new screensavers, the new transparent windows in Aero, the new WindowsKey+Tab feature. My goodness, you HAVE to check out the new tabbing functionality once are up and running on RC1 of Vista when it comes out (you ARE going to upgrade, right?). I think I spent most of last night just tabbing through all my applications. It's a 3d view of your windows from a semi-side perspective, and all the transparencies are still there. Not only that, all the windows are still active. So, let's say I have a flash movie running in one window. If I activate the new tabbing window, the perspective will change AND the movie will continue to play. Very cool stuff!
Anyways, yeah, it's the small things that make me happy. That's not to say there are only small changes. Just you wait and see until you get your hands on all the new security/stability improvements in this version of Windows. In the sense that XP was a big step up from Windows 98, I think Vista is also going to be a big step up from XP. Remember though, this is just my opinion though :P.
I'm sure that many geeks remember the whole Sony Rootkit fiasco that happened not too long ago. I'm sure many people had the seem feeling that I did that it was not going to be the last time we heard about "rootkits" (and that we would probably be hearing even more about them in the future). The work that Mark Russinovich did during the Sony Rootkit debacle is stuff that I still look up to, in a big way. It was yet another data point that helped fuel my desire to learn the internals of Windows and to get a more solid understanding of how everything works.
Anyways, I digress as I'm being a bit tangential.
Some of the cool new features that are starting to pique my interest in Longhorn Server are the new hardware-based virtualization features that are being made possible by technologies like LaGrande (Intel) and Pacifica (AMD). These features could really start to help fuel the fight in IT departments around the industry to leverage virtualization packages to help control various operational costs. In my opinion, this is "A Good Thing." What's interesting though are the holes this technology might open up for various malware authors.
100% undetectable malware? According to this recent article, not only is it possible, but there is already an example out there showing how it can be done. I would be interested to see what some of the brain trust within Microsoft is thinking in regards to stopping these sort of attack vectors. Currently, Vista is set to release and be one of the most secure Windows OS's that we have ever seen. It would be unfortunate if all the work to harden the Windows Kernel would go to waste due to an open attack vector made possible by the new virtualization technologies.
My worry? If not careful, this could undermine the work that is being done to secure Vista in the eyes of the customers. I remember seeing in a previous article or two that there is a "Anti-Malware Technology Team" at Microsoft. I would have to imagine (or perhaps "radically hope" depending on your optimism or pessimism) that this team at Microsoft has started to think about this problem (and hopefully work on it).
As you readers out there may or may not know, I'm a fan of all things games. Heck, you could probably sell me a hunk of plastic if you were able to somehow convince me that it was in fact a game (double-whammy if you were able to convince that it was somehow a football game). This piece of news might help explain why such a little piece of sweet Vista toast can make my evening (although, I must admit, it may have been more "balloon" than it was "toast").
I was in the process of installing Visual Studio 2005 into my VM of Vista Beta 2 this evening when a little nugget popped up to say hello. As Visual Studio was installing, I turned on my XBox 360 to do, well, what we XBox 360 owners do of course (disclaimer: if you do not presently own, or have never owned an XBox 360, it is strongly encouraged that you "waddle right out" and purchase one, post haste [ED: I added the phrase "waddle right out" since the legalese that my "lawyer" (aka "Hugsy, the Bed Time Penguin Pal") came up with was rather drab and boring]).
When the 360's beautiful, glorious, green light came to life and lighted up my life, Vista gave me a little surprise as well. It appears that Vista and the 360 are in fact long lost friends and, to my surprise, Vista shouted with joy when they were finally re-united again. Stargate (aka my Vista box (hey, I've never claimed *not* to be a geek you know)) informed me that not only did he know my 360, but that he would be more than happy to share all his possessions with the 360. The price? Only a few clicks. Heck, that's a price I'm more than willing to pay. After a couple clicks here and there, my Vista machine was ready to share media with my 360 and be inaugurated into our big, happy family. Slap me silly and call me "Mr. Fan-Of-Interconnected-Devices-Who-Are-Easy-To-Setup-And-Share-And-Party-With."
I haven't dug into the depth of this sharing experience yet, but you can bet your bottom dollar that I will start playing around with it this weekend. I'm sorry for such an "over the top" post, but I get so excited when I encounter technology that makes my life better and makes the geek in me cry with joy (and for those who know me, you know I'm large enough to *literally* have an entire geek within me to begin with (dare I say that I, indeed, may actually have a living and breathing geek within my bell-eh (it would certainly explain all the muffled shouting I hear from time to time))).
If only all things in my life could be so easy...
Note: I know that a good amount of this type of functionality is available with Media Center Edition, but alas, I do not have that and this is my first "media sharing" experience with the XBox
I was asked recently what excited me most about the various Vista technologies. While I took a valiant stab at limiting my answer down to a handful of them, the proper answer with the way I feel would have been "what _doesn't_ excite me about the various Vista technologies." Yes, I could get into the various aspects that excite me about Workflow Foundation, Presentation Foundation, and/or Communication Foundation. I will save that for a later blog post though. Save to say, I'm extremely stoked right now about doing some development with all the different Foundation technologies. I think I'm going to do some deep-diving into Workflow Foundation for a while though :).
So, after a hiatus for almost a year (has it really been that long?), I will be spending time again getting into the various Vista technologies that are coming our way. So stay tuned in the future for more Vista updates :).
|