Hello, everybody, I have released the first version of a rudimentary managed wrapper around Transactional NTFS to Code Gallery on MSDN: http://code.msdn.microsoft.com/txfmanaged.
The wrapper in its current form is very basic but should give developers a good idea on how simple it is to use Transactional NTFS from managed code. Feel free to go grab it, play around with it (there are some sample screencasts using the wrapper on the Developer Meet Server Show), and feel free to use it any way you wish.
Enjoy!
A new video has been posted to the Developer Meet Server Show on Channel 9. In this video, we discuss tools and resources you should know about when debugging application compatibility issues. From the description of the video:
"In this video, we sit down (well, stand up actually) with Chris Jackson and Gov Maharaj to chat about debugging application compatibility issues (and some of the things Microsoft has done to mitigate typical app compat issues we've run into)."
Enjoy!
What are sub-queues? Subqueues are implicitly created local queues that are logical partitions of a physical queue. Applications can use subqueues to group messages. Subqueues are implicitly created when opened, are deleted when empty, and have no open handles. Messages cannot be sent to a subqueue. Messages can be received from subqueues, moved between the main queue and its subqueue, or moved between a pair of sibling subqueues.
Subqueues do not have their own properties or state, but share the properties and state of their parent queue. For example, a subqueue does not have its own quota, access control list (ACL), or transactional type. Subqueues share the quota of the main queues. Messages in the subqueues consume the quota of the main queue. Subqueues cannot be created under journal and other system queues, or other subqueues. If journaling is enabled, copies of messages that are removed from a subqueue are sent to the journal queue of the main queue.
So why sub-queues? MSMQ-based applications often require messages to be grouped based on some criteria. Previously, the only way to do this was to create several queues and when a message arrived in the main queue, move the message into a specific queue based on some selection criteria.
Here’s the thing: moving messages between physical queues is a resource-intensive process. If you are wanting to build a high performance application, it is best to try to avoid this resource-intensive process. But if you were building an application based on MSMQ 3.0, this was not very feasible. Now, the new sub-queues feature enables you to logically group messages within a single queue without the need of creating another physical queue.
- MSDN Documentation
- WCF Examples
I have posted a new video to the Developer Meet Server Show on Channel 9. This video discusses common MSI Installer Issues you might run into if deploying an application onto Windows Vista or Windows Server 2008. From the description of the video:
"In this video, we chat with Robert Flaming about application compatibility issues around Windows Installer for Windows Vista and Windows Server 2008. Mark Taylor, from our first Application Compatibility video, sits in as well to fire off some important questions (well, questions that are perhaps more educated than I would ask)."
Enjoy!
What is Windows Process Activation Service? Windows Process Activation Service, also referred to as WAS, is a new component in IIS 7.0 that manages application pool configuration and worker processes instead of the WWW process. This enables the same configuration for both HTTP and non-HTTP sites to be used. Thanks to this separation (and in combination with the new modular architecture of IIS 7.0), you can even host non-HTTP sites without the WWW Service even being installed in the first place.
How does Windows Process Activation Service work? Taken directly from the documentation:
“The WAS process model generalizes the IIS 6.0 process model for the HTTP server by removing the dependency on HTTP. This allows WCF services to use both HTTP and non-HTTP protocols, such as Net.TCP, in a hosting environment that supports message-based activation and offers the ability to host a large number of applications on a given machine."
What scenarios does this enable? Since WAS is not specific to HTTP sites, you can use WAS to host non-HTTP sites as well. But what do we mean by “non-HTTP sites?” Well, simply put, WAS can be used to host sites built on technologies like Windows Communication Foundation, for example. If you are using WCF with WAS, are you limited to listening over HTTP? Not at all. In fact, that is the beauty and power of WAS. You can be hosting a WCF service within WAS that is using a netTcpBinding, netMsmqBinding, etc. As an extension to this, since WAS supports both HTTP and non-HTTP sites, this also means that you can be hosting a service that exposes itself over both HTTP and NET.TCP as well.
Well, at least I found it funny. A coworker brought this to my attention today. I thought it was pretty clever and enjoyed it. Then again, I've known for a while how great Windows Server 2008 is :). Here's an excerpt:
Once I was almost famous. For years, my friends and I were on the front lines: we were the Windows Server 2003 servers that powered Microsoft.com, one of the hottest Web sites in the world. Then, early last summer, everything changed. Quietly, without warning, the new kids took over. Windows Server 2008. Yes, I know, the product’s not even done yet. These were Beta 3 servers, for Pete’s sake. Long way from prime time. But there they were, humming away. No problems. All of ‘em on Windows Server 2008. Except me. The last Windows Server 2003 left at Microsoft.com.
Enjoy!
Hello everyone, and welcome back after our winter break hiatus. We have a lot of cool features to cover in the coming weeks that will especially show off the power of Windows Server 2008 as an application server. So, this week we will kick off the new year by taking a short little peek at the new management API coming in IIS7.
What is new in IIS7 Administration? IIS7 provides a comprehensive managed-code API that allows complete manipulation of the XML configuration files and convenience access to server objects. IIS7 includes Microsoft.Web.Administration, which is a new management API for the web server that enables editing configuration through complete manipulation of the XML configuration files and also provides convenience objects to manage the server, its properties and state. The configuration editing aspect of the API provides programmatic access to read and write configuration properties in the IIS configuration file hierarchy and specific configuration files. The object management aspect of this API provides a series of top-level administration objects for direct management of the server (i.e. sites, application pools, worker processes, etc.).
The management classes reside in the Microsoft.Web.Administration namespace. The classes provide a weakly-typed interface to access configuration sections and convenience objects with properties and methods representing attributes of the configuration (like the path of a virtual directory) or actions to take on the object (like recycling an application pool).
Resources
As part of the "Developer Meet Server" show on Channel 9, I have started a new series of videos on the topic of application compatibility for Windows Server 2008. I will be releasing one video interview per week all the way until launch, so make sure to keep an eye open!
in this first video, we discuss a bunch of topics in this video that are very pertinent to the topic of Application Compatibility on Windows Server 2008, from Session 0 Isolation to IIS 7 to Mandatory Integrity Control, among many others (I will give any of you five geek points if you can work in "Session 0 Isolation" or "Mandatory Integrity Control" into your next dinner party conversation!).
So, make sure to go check it out.
In my previous post, I talked about why I personally love C++/CLI as a solution for managed and native interop. With the launch of Visual Studio 2008, and the include of the new marshalling library, there is on part of the equation missing, I believe. On the C# side of the interop equation, you have http://www.pinvoke.net. When I'm wanting to interop with a given Windows API, there is a very (very) good chance that the wrappers have already been posted onto pinvoke.net. While the marshal_as<> construct of the marshalling library in VC++ 2008 is quite extensible through the use of templates, there is no community site for me to go to in order to retrieve an existing piece of the interop puzzle. That is, until now. With the help of Kate Gregory, there is now a new website for those C++/CLI users out there: http://www.marshal-as.net. If that link doesn't work for you, give it a day or two as this is a new website and DNS is still propagating for it. I have already sent Kate a couple of samples that I had done for my own interop work with Visual C++ 2008 (have I mentioned yet how easy my life is now that I'm using C++/CLI in Visual C++ 2008?). You should pop over and check it out. If you have your own samples that you want to submit, feel free to use the contact form (the email icon) at the website and email your sample to the website. Enjoy!
Howdy all, I will be at Tech Ed Developer this week doing several things. First of all, I will be giving two talks: the first on Application Compatibility in Windows Server 2008 outlining the top issues developers need to be aware of when developing for it, and second, a talk on Transactional NTFS as it pertains to Windows Server 2008.
Also, I will be "manning" the Ask The Experts booth for Windows Server 2008. So if you want to chat about Windows Server 2008 (or just say hello), make sure to drop on by (I will have the hours I plan on working the booth posted there at the booth).
Here's a rundown:
- Monday, 3pm - [INF203] WS08 For Developers - App Compat, Tent 1
- Thursday, 4:30pm - [INF307] WS08 For Developers - Transactional NTFS
I'm also hoping to attend some sessions myself. These will mostly be around C++ (TLA323, TLA302, TLA404, etc.).
So, if you're going to be at the conference, please, drop on by the Ask The Experts booth for Windows Server 2008 and say hello!
James Conard announces over on his blog that our team has just released the Windows Server 2008 Developer Training Kit for Beta 3. Mosey on over and check it out. You can find resources on everything from WCF, to Cardspace, to WF, to Windows Eventing, to PowerShell, to IIS 7, to Transactional NTFS.
So if you are a developer interested in checking out Windows Server 2008, make sure to grab it.
Well, it's official. Beta 3 of Windows Server "Longhorn" has launched. With it, I finally get to release some of the work I've been working on lately.
If you are a developer, I encourage to take a look at "The Top Ways To Light Up Your Apps On Windows Server 'Longhorn'" that was just launched today. It outlines key ways that you, the developer, can leverage all the new and improved enhancements to the Server platform to really make your applications shine in a whole new light.
I encourage any of you with access to Beta 3 to try it out. Windows Server "Longhorn" is quickly shaping up to be the best Server release from Microsoft ever.
I recently posted the second screencast in my new "Developer, Meet Server" screencast series. This latest screencast covers using Transactional NTFS in a service-oriented environment via WCF.
Enjoy!
Andrew recently watched my screencasts and had some great questions that came up around how Transactional NTFS behaves with the file system. So, I figured I would answer them here. So, if you want to see the context around his questions, check out his post.
Here's the paragraph with the questions:
I wonder what happens to files that never get committed - ala a long running transaction. They have to be taking up file space. Jason says not even Windows Explorer knows about them, but they have to exist somewhere. What if they overflow? Is there a way to clean house? If the poop is invisible, does it still stink?
Great observations and questions Andrew! In the case of a long running, open transaction, the files will exist on the disk. Hence, they are certainly taking up space on the disk. If you have 256 MB of free space on your disk, and try to write 257 MB of data to the disk within a transaction, you will get an "out of space" error like you would outside of a transaction.
Is there a way to clean house? Yes, rollback or commit the transaction. On rollback, any changes made within the transaction are rolled back, and then consequently removed form the disk.
But what if my server crashes in the middle of a transaction. Would I have turds floating around the file system? No, you would not. When an NTFS-formatted volume is mounted in Vista or Windows Server "Longhorn", any transactions that are "in-doubt" (the transaction manager is no longer around and we have no clue on the status of the transaction) will be rolled back automatically, effectively removing all the changes from disk. So, in that sense, you would not have any file turds you have to clean up after a system crash (like you would have if you were doing a "temp file -> rename" type of methodology.
Now, as is usual in transaction systems, you have to be _very_ aware and _very_ cautious of long-running transactions as it could cause some long-term problems that you don't want to deal with. So, there certainly are scenarios where Transactional NTFS just is not a good fit. As tends to be the case with all technology, make sure you are using the right tool for the right job :).
Welcome everybody to the inaugural, first ever, grand opening, “opening the doors to the public” celebration day kick-off for the new “Developer, Meet Server” screencast series. I am your sincere, humble (as ever), gracious and wonderful host/chef/tour guide Jason Olson.
In this series of screencasts we will be diving through all the various cool developer-oriented enhancements on the Windows Server "Longhorn" platform that developers can leverage to build robust and powerful applications. Specifically, in this first screencast, we will take a quick peek at Transactional NTFS. In future screencasts, we will go into a more detailed look at Transactional NTFS, so think of this as a quick introduction.
The screencast can be found in one of the main tubes on the Intarw3b known as the ever-lovely Channel 9: http://channel9.msdn.com/Showpost.aspx?postid=289816.
So sit back, grab a cup of coffee/tea/InsertYourFavoriteLocalTastyBeverageHere, and enjoy this screencast.
Hello everyone, and welcome back to the “Feature Of The Week” newsletter. This week, we will revisit a prior Fear Of The Week technology that we discussed in order to show off a cool capability of that technology. If you are a developer wanting to mess around with TxF (Transactional NTFS), you should especially read on :).
Did You Know? Did you know that on Vista, Transactional NTFS will work with an NTFS-formatted USB Flash Drive? It’s true! If you want to play around with Transactional NTFS and how it can interact with SQL, for instance, but you only have one hard drive, you don’t have to worry about Secondary Resource Managers and all that hubbub, gizmos, doodads, and what-cha-ma-call-its. Just pop in that NTFS-formatted USB Flash Drive, and play around until your heart’s content.
Since the USB Flash Drive is not the system drive (well, we hope it’s not the system drive in this case :P), you can circumvent the need for secondary Resource Managers when wanting to test out interoperability scenarios with Transactional NTFS (like TxF + WCF, TxF + SQL, TxF + MSMQ, etc.).
Happy Coding :).
Generic Transactional NTFS Links...
Documentation
Videos/Webcasts
Blogs
Hello everyone, and welcome back to the “Feature Of The Week” newsletter. This week’s newsletter will be short and sweet. We will go over the new Application Recovery and Restart API. The API itself is very small and very simple to use (which, in my opinion, is “A Good Thing” TM).
What is Application Recovery and Restart? Application Recovery is a set of technologies that give applications the opportunity to control what actions are taken on their behalf by the system when they (the application) fail.
If the application registers a callback with the RegisterApplicationRecoveryCallback function, the system will execute the recovery callback on application failure. The application can that take action to do things like attempting to recover documents. If the application has not registered for recovery, the system attempts to diagnose and report the problem.
If the application registered a callback with the RegisterApplicationRestart function, the system will automatically restart the application after the recovery or diagnosis is completed. If the application has not registered for restart, the system will simply close the application.
Documentation
Screencasts
I recently came across a bunch of stuff done by Daniel Moth, a Developer Evangelist for Microsoft UK. If you are a managed developer and want to see the new stuff you can leverage on Vista, be sure to check out his stuff.
If you want to digest some of these new features in Screencast format, be sure to check out some of these screencasts that Daniel Moth has also done:
Enjoy! And thanks go to Daniel for all of these :).
Hello everyone, and welcome back to the “Feature Of The Week” newsletter. In this week’s edition of the “Longhorn Server Feature of the Week”, we will be taking a look at a new feature in Vista and Longhorn Server: the Restart Manager.
What is the Restart Manager? The primary reason software updates require a system restart during an installation or update is that some of the files that are being updated are currently being used by a running application or service. The Restart Manager enables all but the critical system services to be shut down and restarted and guarantees that blocking DLLs and resources are unloaded. This frees files that are in use and allows installation operations to complete.
Using the Restart Manager DLL, an installer can use the Restart Manager to register files that should be replaced during the installation of an application or update. Then during a subsequent update or installation, the installer can use the Restart Manager to determine which files cannot be updated because they are currently in use. Installers can direct the Restart Manager to shutdown and restart applications or services based on the file in use, the process ID (PID), or the short-name of a Windows service.
What scenarios does this enable? The Restart Manager API can eliminate or reduce the number of system restarts that are required to complete an installation or update.
Notes – Applications that use Windows Installer, version 4.0, for installation and servicing automatically use the Restart Manager to reduce system restarts. Custom installers can also be designed to call the Restart Manager API to shutdown and restart applications and services.
Resources
Hello everyone, and welcome back to the “Feature Of The Week” newsletter. This week, we take quick dive into some of the enhancements that have been made to Windows Error Reporting (WER) in Vista and Longhorn Server.
What is Windows Error Reporting (WER)? Windows Error Reporting enables users to notify Microsoft of application faults, kernel faults, and unresponsive applications. Microsoft can use the error reporting feature to provide customers with troubleshooting information, solutions, or updates for their specific problems. Developers can use this infrastructure to receive information that can be used to improve their applications.
What’s new in Windows Error Reporting? Windows Error Reporting isn’t actually new to Vista and Longhorn Server. However, there are some pretty cool enhancements to Windows Error Reporting that you can find in Vista and Longhorn Server. Developers can use the new API to create reports for many types of events, not just applications crashes and hangs. Also, developers can now create custom reports, customize the reporting user interface, and submit reports to Microsoft. Using Windows Quality Online Services, they then can access their report data, create solutions, and deliver these solutions to their users.
Resources
Hello everyone, and welcome back to the “Feature Of The Week” newsletter. This week, we will be looking at the robust replacement for boot.ini in Windows Vista and Longhorn Server: Boot Configuration Data.
What is Boot Configuration Data? Boot configuration data (BCD) provides a store that is used to describe boot applications and boot application settings. The objects and elements in the store effectively replace Boot.ini. The data in the store can be managed by a command-line utility, bcdedit.exe, or using the classes implemented by the WMI provider.
How does Boot Configuration Data work? When you start your computer, the first code that executes is the BIOS. The BIOS reads the master boot record (MBR) from the boot device and transfers control to the boot code stored in the MBR. The boot manager reads the boot entries from the Boot Configuration Data store so they are available to the loader and displays a boot menu to the user. The boot environment provides a native API for primitive graphics and other system support. Boot applications are pieces of code that are located on a boot device and run in the boot environment.
The following are examples of application objects:
• The Windows Boot Manager
• The Windows OS Loader
• The Windows Memory Tester
Example – Now, with the absence of the boot.ini file, this is how you would enable Kernel debugging on a system as a boot option: “bcdedit /debug true”
Resources
Hello everyone, and welcome back to the “Feature Of The Week” newsletter. This week, we will take a short little peek at the new Windows Remote Management infrastructure in Vista and Longhorn Server.
What is Windows Remote Management? Windows Remote Management (WinRM) is the Microsoft implementation of the WS-Management protocol, a standard SOAP-based, firewall-friendly protocol that allows hardware and operating systems from different vendors to interoperate.
The WS-Management protocol specification provides a common way for systems to access and exchange management information across an IT infrastructure. WinRM and IPMI (Intelligent Platform Management Interface), along with the Event Collector, are components of the Windows Hardware Management features.
What scenarios does this enable? Using WinRM scripting objects, the WinRM command-line tool, or the Windows Remote Shell command line tool, you can obtain data from local and remote computers. You can also obtain hardware data from WS-Management protocol implementations running on non-Windows operating systems. WinRM can also supply standard WMI (Windows Management Instrumentation) data, but it handles remote connections via the SOAP-based WS-Management protocol rather than DCOM.
Resources
Hello everyone, and welcome back to the “Feature Of The Week” newsletter. In this fifth installment, we will take a short little peek at a series of enhancements coming in Vista and Longhorn Server to the Task Scheduler.
What is the Task Scheduler API? The Task Scheduler API is an API that enables you to automatically perform routine tasks on a chosen computer. The Task Scheduler does this by monitoring whatever criteria you choose to initiate the tasks (referred to as triggers) and then executing the tasks when the criteria is met.
What enhancements have been made in 2.0 over 1.0? The following Task Scheduler 2.0 changes are introduced in Windows Vista:
- Isolating user actions in separate sessions
- Credentials managements using new security services (S4U and CredMan)
- Removed limitations on the number of registered tasks
- API changes:
- Scripting support: interfaces are now derived from IDispatch, providing full support for script development
- Task Scheduler schema: allows you to create/manage tasks through XML-formatted docs.
- New Triggers: new time, calendar, and event triggers. All triggers support repetition, delay, start, and stop boundaries.
- Task settings: can now prioritize tasks, define multiple instance policies, start a task only when it is available or only if the network is available, restart on failure, and set an execution time limit
- New actions: a task can be scheduled to send an email, show a message box, start an exe, or fire a COM handler.
What scenarios does this enable? The Task Scheduler can be used to execute tasks such as starting an application, sending an email, or showing a message box. Tasks can be scheduled to execute:
- When a specific system event occurs
- At a specific time (even on a daily, weekly, monthly, or monthly day-of-week schedule)
- When the computer enters an idle state
- When the task is registered
- When the system is booted
- When a user logs on
- When a Terminal Server session changes state
Hello everyone, and welcome back to the “Feature Of The Week” newsletter. As mentioned last week, this week we will look at the component that enables new technologies like Transactional NTFS and the Transactional Registry in Vista and Longhorn Server: the Kernel Transaction Manager.
What is the Kernel Transaction Manager? The Kernel Transaction Manager (KTM) is a transaction management service. It makes transactions available as kernel objects and provides transaction management services to system components such as Transactional NTFS (TxF).
KTM is scalable; it will work on both large-scale applications and small-scale applications. KTM provides isolation across all types of operations, allowing a consistent view of data. You can use the transaction scope to “bracket” the sections of your application, providing atomicity for some operations and not others. KTM helps with concurrency problems. It does not affect performance, and it is available everywhere. KTM can be used to provide transactions for file management scripts and backups. During backups, KTM can provide a consistent snapshot between the registry, file system, and databases.
What scenarios does this enable? The Kernel Transaction Manager enables the development of applications that use transactions. The transaction engine itself is within the kernel, but transactions can be developed for kernel- or user-mode transactions, and within a single host or among distributed hosts. Transaction-aware applications have complete flexibility in the granularity with which they use transactions. For example, it is possible for an application to access some files under transactional control and access others using operations that are not part of a transaction.
Resources
Hello everyone, and welcome back to the “Feature Of The Week” newsletter. In this week’s newsletter, we will continue on the “Transactions Bandwagon” by discussing the Transactional Registry. This newsletter will be short and sweet as, truth be told, this feature still needs to have a bit more documentation than it has today.
So, sit back, relax, and enjoy the Longhorn Server Feature Of The Week!
What is the Transactional Registry? The Transactional Registry (TxR) is an enhancement to the Registry that allows all registry operations to be performed within the scope of a transaction. This is made possible by a new transaction infrastructure in the kernel that allows operating system services to participate in transactions via the new Kernel Transaction Manager.
What scenarios does this enable? The Transactional Registry (TxR) is also fully capable of communicating with the MS Distributed Transaction Coordinator, meaning that TxR operations can participate in transactions that are leveraging any other Resource Manager that DTC is capable of working with. What does this mean to you? It means that you can now have an application that not only encompasses database calls, operations to the file system, as well as operations to the Registry.
Preview: Next week we will take a look at the technology that makes the Transactional NTFS and Registry possible: the Kernel Transaction Manager. Stay stuned!
[Update: Added link to RDC Whitepaper and blurb about RDC dogfooding in DFS Replication (thanks cfsbloggers)]
Hello everyone, and welcome to the second installment of the “Feature Of The Week” newsletter. This week, we will take a short little peek at a feature coming in Vista and Longhorn Server called Remote Differential Compression.
So, sit back, relax, and enjoy the Longhorn Server Feature Of The Week!
What is Remote Differential Compression? Remote Differential Compression (RDC) allows applications to synchronize data between two computers in an efficient manner. The synchronization efficiency is made possible by using compression techniques to minimize the amount of data sent across the network.
What makes RDC different from other differencing mechanisms? RDC is different from patching-oriented differencing mechanisms, such as Binary Delta Compression (BDC), which are designed to operate only on known versions of a single file. RDC makes no assumptions about file similarity or versioning. Because differences between files are computed on the fly, RDC is ideally suited for synchronizing files that are different or have been updated independently.
What scenarios does this enable? RDC is suitable for applications that move data across a wide area network where the data transmission costs outweigh the CPU cost of signature computation. RDC can also be used on faster networks if the amount of data to be transferred is relatively large and the changes to the data are typically small.
Dogfooding RDC. It's important to note that we are actively dogfooding RDC internally. One of the current examples of that dogfooding is that RDC is actively being used for DFS Replication, available on Windows Server 2003 R2.
Resources
- Documentation
- Code
- If you have the latest Windows SDK installed, you can find a simple Client/Server example using RDC in the following directory: %INSTALL_DIR%\Samples\winbase\RDC
Enjoy!
Hello everyone, and welcome to the “Feature Of The Week” newsletter. In the Feature Of The Week Newsletter, we visit a new feature coming in Longhorn Server every week. The goal is that these are nice, short, and efficient to minimize the amount of effort required by you, the reader. With every Feature Of The Week, we will provide links to more resources in case you want to spend more time diving into the feature covered.
So, sit back, relax, and enjoy the Longhorn Server Feature Of The Week!
What is Transactional NTFS? Transactional NTFS is an enhancement to NTFS that allows all file operations on an NTFS file system volume to be performed within the scope of a transaction. This is made possible by a new transaction infrastructure in the kernel that allows operating system services to participate in transactions via the new Kernel Transaction Manager.
What scenarios does this enable? Transactional NTFS is also fully capable of communicating with the MS Distributed Transaction Coordinator, meaning that TxF operations can participate in transactions that are leveraging any other Resource Manager that DTC is capable of working with. What does this mean to you? It means that you can now have an application that not only encompasses database calls, but also operations to the file system as well (think of Document Management solutions). With transaction enhancements being made to the new SMB 2.0 protocol, you can also now contain distributed file operations within a transaction as well.
Resources
- Documentation
- Videos/Webcasts
- Blogs
Enjoy!
Howdy all y'all. I'm hoping to get some feedback from all of you. Basically, I'm hoping to get a "reader's braindump" on what development features you are looking forward to most in Vista/Longhorn Server. So, what development features are you looking forward to most in Vista/Longhorn Server? Feel free to just leave a comment (nice and easy-like) :).
Jim Johnson has a new post about how to use TxF with the new model in Vista RC1. You will obviously need the RC1 bits to mess around with it.
I'm hoping to expand on this topic in the future to show some code samples of using it not only by itself, but also using it with SQL and the like. I'll also have some "context" posts around that describing the architecture and relationships around TxF so you can better understand the articles.
Until then, go check out Jim's post :).
A friend of mine made a good comment on my last post about TxF: "what the heck does TxF do and why should developers care?" I hope to answer that question in this post. If you've never heard of TxF, read on. If you have and still don't "grok it", read on. If you just happen to love the wonderful language that is expelled from my tranquil appendages as they dance over the keyboard, then by all means, read on as well :).
So, what is TxF? TxF is the "nick name" for a new technology coming with Vista and Longhorn Server: Transactional NTFS. You may have read about or used logging file systems in the past, and you may have used journaling file systems in the past as well. But TxF is a fully-transactional file system, all built on top of NTFS. And when I say "fully-transactional", I mean it supports full ACID transactions on the file system itself.
Why should you care about TxF? You should care about TxF because the transactions that operations are contained in are not only specific to the file system. When working with TxF, there is full support for the Distributed Transaction Coordinator as well. This means that now I can coordinate transactions between the file system and SQL. Heck, this also means that I can coordinate transactions between the file system and Windows Communication Foundation.
Just stop for a second, and think about the scenarios that this opens up to us as developers. Let's look at a Document Management System, for example. If I were to submit a new version of a file into the system, the database may have to store a hash of the file as well as a pointer to the file on the file system. The problem is that we have to now guarantee that the hash of the file in the database in identical to the version of the file on the file system (I'm of course assuming that we aren't just shoving everything into blob fields).
If we update the file on the file system first, there are at least two situations that can occur that we might have to deal with: 1) If someone tries to pull down the file before the database has been updated, then the hash comparison will fail and the file will falsely be identified as corrupt. 2) If the database update fails for any reason at all, we now have to write extra code to roll back the file manually, and in the meantime, users pulling down the file could have it falsely identified as corrupt again.
So, let's change the order. Let's say we update the database first and then the file. Unfortunately, we're not in a better position, only reversed from before. Either way we go, there is more code for us to write if we want a stable system. This is where TxF can come in to make our systems more stable and robust. We can use TxF's ability to coordinate with DTC to manage all of this for us.
First of all, it would be completely Atomic. By using DTC for our database update, and TxF for our file update, they can coordinate with each other to guarantee that either they both succeed or none of them do. That's right, if the database update fails, the file system update will be rolled back, and vice versa. The transaction is also Consistent. At the same time, the file system update is also Isolated. After we update the file and before we commit the transaction, every one outside of the transaction will still see the old version of the file (how cool is that?!?!?). And the system is Durable. If we commit the transaction and the system crashes, when it restarts, the file will be guaranteed to be updated. And all of this is handled by TxF (well, handled by the new Kernel Transaction Manager, to be specific).
That's just the beginning of the story though. Imagine this on a distributed scale. We can use TxF to manage file operations across multiple machines. We can also coordinate TxF with WCF using System.Transactions :D. So, let's say I wrote a Music Download service. As a user, I want to ensure that my credit card is not charged unless I download the file completely. This is a scenario that becomes a lot easier with TxF. If my credit card was not able to be charged, I should be not be able to keep the file. On the other hand, if I was not able to download the file successfully, my credit card should not be charged (disclaimer: there are some obvious security holes to work through in this example :P).
By adding transactional support into the file system, we are also hoping to make your lives easier in the long run. You no longer have to write extra code to self-manage the file system when dealing with distributed systems. By leveraging TxF and System.Transactions, all you basically need to do is code against "The Happy Path." You just have to ensure that if anything fails, all of it is rolled back (which is ultimately handled by the system). There are a couple of sayings I like to keep in mind as a developer: 1) The fastest running code is code that doesn't run at all (I believe I got this one from Rico M.) and 2) Code that isn't written is code that doesn't have to be tested. Easier lives through transactions :).
It's not all glitz and glamour though. In my last post on TxF, I discussed some of the "down side" of TxF as the API exists today. There were two "features" that are not implemented that I didn't mention. Mainly, at this time, we do not have support for transactions on streams, or transactions when dealing with encryption. The latter, I am told, is because of some interesting security challenges that we need to iron out first.
One of the challenges I have ahead of me as a Technical Evangelist is "spreading the word" of TxF. Even before I joined Microsoft, I had only heard about it because of a couple of things: 1) the job posting for the job I interviewed for and 2) A video about TxF made by Channel 9. There really isn't a lot of information out there on TxF. And the information you can find is largely based on Vista Beta 2 or earlier, meaning, it is now quite out of date.
When chatting with Carlos about it, he said (and I quote): "I read a lot of blogs and that's the first I've ever heard of it." I'm pretty sure he's not the only one that would have this same reaction. Time to spread the word, my friends :). I only hope I can get a fraction of you out there as excited about TxF as I am. If I can do this, then hopefully the technology will speak for itself and word of it will spread fairly quickly.
[Usual Disclaimer: my opinions in no way reflect or represent the opinions of my employer]
You'll probably hear me start cheering for TxF after a couple of posts because it is a great technology, but at this point in the ballgame, let's take a step back and take a sobering look at the state of TxF as it exists in post-Beta 2 Vista.
As I've been getting up to speed with TxF, I've realized that there is very little up-to-date information on TxF available (except for an internal presentation that was given at an internal conference). Most likely, any blog posts you'll find on the internet regarding TxF and how to use it are no longer accurate because of the newly revamped API. The extent of up-to-date information even available internally is constrained to a single presentation by the PM and the header files for the various APIs. Of course, it is one of my responsibilities to contribute to the amount of content that is available, so I suppose you could consider it job security if I do my job right :).
TxF itself, is a VERY cool technology. And to put together a managed code example of how to use the new KTM to manage transactions across files simply on the local file system is nearly trivial (you can expect to see a Screencast on Channel 9 from me on just how to do this). However, the truly interesting and compelling scenarios (I believe) don't exist simply with only files on the local file system. To me, the interesting scenarios are when you are managing transactions that span SQL and TxF, or WCF and TxF, or heck, even SQL + WCF + TxF. However, when we get to these scenarios, the "story" breaks down a bit.
If you are a C/C++ developer, you're fine, you're clear, you're out of the woods, you've been cleared for a safe landing, you can use this stuff full-force out of the box. If you are a managed code developer on the other hand, buckle your safety belt folks, we're going in for an emergency landing. To me, this part of the story, frankly, SUCKS (and trust me, we know it sucks). You heard it correctly folks, as of the post-Beta 2 API, there is no longer any Managed API into TxF. If you want to use TxF from C# (and have it work with SQL/WCF via DTC), you will be following the brown-bricked road into P/Invoke and Marshal.QueryInterface land.
The removal of the managed API can partially be attributed to the removal of the implicit transaction model, which brings up another unfortunate side-effect. When the implicit transaction model existed, you could leverage TxF with little to no code changes. You could basically instantiate a TransactionScope, have KTM enter the transactional context (EnterTransactionScope/ExitTransactionScope in kernel32.dll), and then utilize the normal File class APIs. Since the transaction model was implicit, the files would (you got it) _implicitly_ be enrolled in a KTM transaction that is being coordinated with the TransactionScope.
With the implicit model having been ousted now, there are a series of new transacted API calls that exist in kernel32. For example, there is now a CreateFileTransacted, OpenFileTransacted, etc. and they all have almost the exact same interface as their corresponding un-transacted siblings (the only real difference being that the *Transaction API calls accept some additional transaction-oriented parameters). As you can see, it is very much an explicit programming model now. Unfortunately, this means that if you have an existing product, you will have to change any file system calls necessary to leverage the new Transacted APIs. It's not too bad, but it still sucks in my humble opinion. Of course, for any managed code developers out there, it sucks more because instead of using the File class in System.IO, you now have to P/Invoke into kernel32 to get at all the Transacted APIs.
In my opinion, the challenge this presents us as a company is that if we want our managed developers out there to use this, we absolutely CAN'T expect them to use it as it exists today. I think if we tried to do that, adoption would fall flat on its face. My current thoughts are that we can perhaps release a publicly-available community wrapper (maybe on GotDotNet, CodePlex, or something else) that people can use to easily interact with TxF with very little effort. I also believe that for even that to be successful, it can't be a "totally new" API. We should leverage the existing look and feel of APIs like System.Transactions and System.IO.File to provide the TxF functionality. I'd imagine this is one effort that I'm going to try to champion and to get out there for all of you. Heck, if not for you, I'll just do it selfishly just for me since I don't want to be constantly context-switching down into P/Invoke, COM madness when writing my screencasts, demos, and other TxF code.
Don't get me wrong, there has been a lot of cool work done. And all of the reasons I've heard on why the implicit model was "broken" (or at least that implementation of an implicit model) make absolute sense. It is just another example of what you have to go through when you have to worry about backwards compatibility, app-compat, etc. Yes, in this case, it has given us a bit more work to do. But I can live with that :).
Why am I making this post? Am I a pot-stirrer? I don't believe so. I simply want to get the truth out there so you all know that there are some warts we need to remove on the product. More importantly, we are recognizing these warts and are actively engaged at trying to remedy them. I truly hope this does not dissuade you from trying it out as it is certainly very cool stuff and shows a lot of potential moving forward (which I hope you'll see as I dive into it with all of you :D).
|