Code Monkey home page Code Monkey logo

txfilemanager's Introduction

Transactional File Manager is a .NET Standard 2.0 library that supports including file system operations such as file copy, move, delete, append, etc. in a transaction. It's an implementation of System.Transaction.IEnlistmentNotification.

This library allows you to wrap file system operations in transactions like this:

// Wrap a file copy and a database insert in the same transaction
IFileManager fm = new TxFileManager();
using (TransactionScope scope1 = new TransactionScope())
{
    // Copy a file
    fm.Copy(srcFileName, destFileName);

    // Insert a database record
    db.ExecuteNonQuery(insertSql);

    scope1.Complete();
} 

Current features

Support the following file operations in transactions:

  • AppendAllText: Appends the specified string the file, creating the file if it doesn't already exist.
  • Copy: Copy a file to another file.
  • Delete: Delete a file.
  • Move: Move a file.
  • CreateDirectory: Create a directory.
  • DeleteDirectory: Delete a directory.
  • MoveDirectory: Move a directory.
  • Snapshot: Take a snapshot of the specified file. The snapshot is used to rollback the file later if needed.
  • WriteAllBytes: Write the specified bytes to the file.
  • WriteAllText: Write the specified text content to the file.

If you have any suggestions for enhancements or bug reports please use the Issues list. Better yet, if possible join this project and contribute.

This library is available as a NuGet package.

This started out as a blog post. It was hosted on CodePlex and migrated to GitHub in 3/2020.

Additional contributors: @gvas, AirBreather.

Quick Start

  1. Add a reference to TxFileManager
dotnet add package TxFileManager
  1. Start writing code
IFileManager fm = new TxFileManager();
using (TransactionScope scope1 = new TransactionScope())
{
    // Copy a file
    fm.Copy(srcFileName, destFileName);

    scope1.Complete();
} 

Frequently Asked Questions

How do I reference this library?

The recommended method is to add a NuGet reference:

dotnet add package TxFileManager

Or Use Visual Studio's Manage NuGet Packages to add. Search for "TxFileManager".

Can I reuse instances of TxFileManager?

It's not expensive to create new instances of TxFileManager as needed. There's a bit of overhead (like creating instances of any small class) but not much.

On the other hand, it's totally safe to re-use the same instance for multiple transactions, even nested transactions.

Is TxFileManager Thread Safe?

Yes - it's been tested for that.

Which IsolationLevel's are supported?

Regardless of the specified IsolationLevel, the effective IsolationLevel is ReadUncommitted.

How does TxFileManager work?

See Chinh's blog post: Include File Operations in Your Transactions Today with IEnlistmentNotification

Where are temporary files/directories kept?

By default, the path returned by Path.GetTempPath() is used to keep temporary files/directories used by TxFileManager. However, you can override that and have TxFileManager use another temp path:

IFileManager fm = new TxFileManager(myTempPath);

How do I contribute to the project?

  • Fork the repo
  • Create a branch such as my-new-feature
  • Make your change/fix and associated unit tests and commit
  • Run the tests (dotnet test)
  • Open a Pull Request (PR) to the "master" branch

Notes: A release branch will be created out of the master branch when a release is made. The master branch will contain the latest features being added/tested for the next release.

Version 1.4 (released 3/2020)

  • Add support for custom temp paths to address issues with file/dir operations accross filesystems
  • Fix for resource leak in TxFileManager._txEnlistment
  • Target .NET Standard 2.0
  • Additional testing for .NET Core on Ubuntu
  • Additional stress testing both on Windows and Ubuntu
  • Created Github workflow to automatically build/test on ubuntu
  • Added FxCop static analysis

Version 1.5.0.1 (released 7/21/2021)

Please suggest your features using the Issues list.

txfilemanager's People

Contributors

chinhdo avatar developermj avatar gvas avatar mugsy0815 avatar remye06 avatar zungmou avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

txfilemanager's Issues

Possible Enlistment leak?

From CodePlex archive:

Hi chinhdo,
It is a great project, I've learn a lot from it.
I have some concerns as follows:
I found it uses File.Copy() to back up for recovery, but File.Copy() is not atomic.
If it is copying the backup while the system is shutting down, the backup file could be corrupted and can't be used for recovery.
In TxFileManager.cs EnlistOperation()
For each thread and each transaction scope, there is a enlistment, when would a specific enlistment be removed from _enlistments? Would there be more and more items in _enlistments as time goes by?
Thank you.
Thread #445077 | Message #1049098 | 2013-05-27

Hi Enjaychang:

Re (1), that's a limitation of this library. It relies on the OS to do the actual file operations. The next step would be to create a file system driver or something of that nature and that would be a very big project outside the scope of this library. There may be incremental things we can add to improve the chance that transaction integrity is preserved in the event of errors.

Re (2), let me look into that. That sounds like it could be a potential memory leak issue there.

Thanks.
Thread #445077 | Message #1053342 | 2013-06-05

Move (file rename) transaction is not rollbacked when program is exited with Ctrl+C (console application)

,NET 5.0

Source:

namespace NTFSTransactions
{
    class Program
    {
        static void Main(string[] args)
        {
            DirectoryInfo folder = new DirectoryInfo("testFiles");
            FileInfo[] files = folder.GetFiles();
            AtomicRename(files.ToList());
        }

        static void AtomicRename(List<FileInfo> files)
        {
            IFileManager fm = new TxFileManager();
            using (TransactionScope transactionScope = new TransactionScope())
            {
                foreach (var file in files)
                {
                    fm.Move("testFiles/" + file.Name, "testFiles/" + file.Name + "_sss");
                    Thread.Sleep(1000);
                    Console.WriteLine(file.Name + " renamed");
                }
                transactionScope.Complete();
            }
        }
    }
}

Rollback works if I return from the function before transactionScope.Complete(); is called.
But it doesn't work if the application is closed (Ctrl+C or Environment.Exit(0));
Is there any way that it could roll back if application is closed (process ends)?

WriteAllText without Encoding uses Encoding.Default instead of Encoding.UTF8

We recently upgraded our packages and with that pulled in 103c685.

Before that commit, the WriteAllText operation used File.WriteAllText without encoding, which uses Encoding.UTF8.
After that commit, the WriteAllText that we use calls a new overload and provides Encoding.Default.
This means that our files are now created in ANSI instead of UTF8.
We hadn't noticed the change because the method we use didn't change, and we assumed the implementation would be the same.
I don't think it's a good idea to use different defaults than .Net, either (as the IFileManager interface seems to mimic the System.File operations).
We can fix our code by providing Encoding.UTF8 in our calling code, but it seems like a step back.

TxFileManager.cs

        public void WriteAllText(string path, string contents)
        {
            WriteAllText(path, contents, Encoding.Default); // this should use Encoding.UTF8 to comply with default .Net implementation
        }

Also, when WriteAllText is called and we are not in a transaction, the encoding is ignored when calling File.WriteAllText (so ironically, when we're not in a transaction our files are created the way we want it).

TxFileManager.cs

        public void WriteAllText(string path, string contents, Encoding encoding)
        {
            if (IsInTransaction())
            {
                EnlistOperation(new WriteAllTextOperation(GetTempPath(), path, contents, encoding));
            }
            else
            {
                File.WriteAllText(path, contents); // this should pass on the encoding as well
            }
        }

can "multiple file move operation rollback" move back as many files as it can instead of stopping on first failure?

Hello,
Scenario:
I want to move 5 files from one directory to another transactionally..

while debugging code on visual studio,

  • In trx scope, after moving 4 files, i throw an exception
  • i pause before getting out of trx scope, and delete one of 4 already moved files from filesystem manually,
  • I expect the library to move 3 files back successfully, but as far as i see, it stops after first failure.. (in my case only 1 file was moved back, 2 of them stayed in destination)

Is it possible for it to move back files with best effort?

SQL transaction not closed when rolling back a file operation fails

We're using ChinDo TxFileManager to be able to rollback file operations when we're rolling back our SQL transaction. This works fine, except when rolling back a file throws an exception (because the file is locked or no longer exists). When that happens, the SQL transaction is kept alive even after we leave the scope, and there's no Transaction left to dispose.

I've reproduced the problem in a small Console Application, where I move the file without the TxFileManager to get the rollback to fail.
I let the console run for a minute afterwards to replicate the behaviour we have in our web application.
As long as the application is alive, the transaction is not rolled back.

using System;
using System.Data.SqlClient;
using System.IO;
using System.Threading;
using System.Transactions;
using ChinhDo.Transactions;

namespace TransactionRollback
{
    class Program
    {
        static void Main(string[] args)
        {
            Test();
            Thread.Sleep(60 * 1000);
        }

        public static void Test()
        {
            File.WriteAllText(@"C:\temp\txFiles\MyFile.txt", "test"); // generate a test file.
            try
            {
                var scope = new TransactionScope(TransactionScopeOption.Required, new TransactionOptions { IsolationLevel = IsolationLevel.ReadCommitted });
                try
                {
                    // start a connection within the TransactionScope and execute a command that locks a row.
                    using var connection = new SqlConnection("Data Source=.;Initial Catalog=MyDatabase;Integrated Security=true");
                    connection.Open();
                    var command = connection.CreateCommand();
                    command.CommandText = @"Update Document SET Path='C:\temp\txFiles\MyFile2.txt' WHERE Id=1";
                    command.ExecuteNonQuery();

                    // use the transactional FileManager (from ChinDo) to move the file we created.
                    var fileManager = new TxFileManager();
                    fileManager.Move(@"C:\temp\txFiles\MyFile.txt", @"C:\temp\txFiles\MyFile2.txt");

                    // delete the moved file without the transactional FileManager so the rollback will fail.
                    File.Delete(@"C:\temp\txFiles\MyFile2.txt");
                }
                finally
                {
                    try
                    {
                        // this throws an exception because it tried to rollback a file that doesn't exist.
                        // this leaves the actual SQL transaction alive.
                        scope.Dispose();
                    }
                    catch (Exception rollbackException)
                    {
                        Console.WriteLine(rollbackException);
                    }
                }
            }
            catch (Exception exception)
            {
                Console.WriteLine(exception);
            }
        }
    }
}

I don't think the rollback should throw errors when it fails to perform a rollback action, because that messes up the distributed transaction.

My current solution is to get the SPID before I rollback, and if the rollback fails I send a command in a new connection to kill that SPID, but that seems sketchy at best.

Directory/File Copy/Move Cancel Support

Can we get versions of all of the methods with cancellation support (cancellation tokens or something similar)? So that I can cancel a file or directory move/copy in progress?

Multiple File Upload Issue

Using txFileManager to upload multiple file upload at once.
The File Upload gets stuck when tried to upload multiple files.
In incognito mode it works fine but in normal chrome browser it doesn't work fine.
After clearing the cache it works fine. I am using version 1.3 is this issue resolved in latest version.
How to resolve this cache issue with file upload?

NullReferenceException with async code

I am using this project to save files to a file system in a transaction with an ef core database transaction. My code is leveraging multiple async methods within services along with ef core async methods. The code fails upon calling of the transaction scope objects Complete method. After pulling the code and reviewing I found that removing the [ThreadStatic] attribute in the TxFileManager class on the _enlistments dictionary to resolve the issue for me.

After thinking through this I'm not sure why the ThreadStatic attribute would be required. I would think that because transactions can run across multiple threads that we would want to keep track of the enlistments the same way.

I can create a PR for this work if anybody finds this valuable. TIA.

DeleteDirectoryOperation fails if temp directory is on a different device.

Issue copied from CodePlex:

Firstly, thank you for TxFileMgr, it's amazing, and we're using it to do good things. Unfortunately, I managed to find a bit of a stumbling block recently when it comes to deleting directories.

The use of GetTmpFileName in DeleteDirectoryOperation.cs may end up with a path that's on a different device to the directory we're deleting. Consequently, the subsequent move operations fail, as directories cannot be moved across filesystem boundaries¹.

A good fix for this would be allowing a temp directory to be provided as an optional argument on TxFileMgr object creation, allowing the application to specify where temporary files should be housed. This should default to querying GetTmpFileName if no argument is provided.

Note that this bug only affects directories; as TxFileMgr copies rather than moves files, they are unaffected by filesystem boundaries.

Many thanks!

~ Paul

¹ At least on Linux.
Hi Paul, Thanks!

Yes I think providing an option to set the Temp folder would work for this issue. I'll work on that... unless anyone else wants to take it.
2014-11-04
You are awesome. Thank you! :)
2014-11-04
Active
#20 | Created 2014-11-03 | Updated 2017-12-14

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.