Asynchronous locking based on a key
I’m attempting to figure out an issue that has been raised with my ImageProcessor library here where I am getting intermittent file access errors when adding items to the cache.
System.IO.IOException: The process cannot access the file ‘D:\home\site\wwwroot\app_data\cache\0\6\5\f\2\7\065f27fc2c8e843443d210a1e84d1ea28bbab6c4.webp’ because it is being used by another process.
I wrote a class designed to perform an asynchronous lock based upon a key generated by a hashed url but it seems I have missed something in the implementation.
My locking class
Usage — within a HttpModule
Can anyone spot where I have gone wrong? I’m worried that I am misunderstanding something fundamental.
The full source for the library is stored on Github here
6 Answers 6
As the other answerer noted, the original code is removing the SemaphoreSlim from the ConcurrentDictionary before it releases the semaphore. So, you’ve got too much semaphore churn going on — they’re being removed from the dictionary when they could still be in use (not acquired, but already retrieved from the dictionary).
The problem with this kind of «mapping lock» is that it’s difficult to know when the semaphore is no longer necessary. One option is to never dispose the semaphores at all; that’s the easy solution, but may not be acceptable in your scenario. Another option — if the semaphores are actually related to object instances and not values (like strings) — is to attach them using ephemerons; however, I believe this option would also not be acceptable in your scenario.
So, we do it the hard way. 🙂
There are a few different approaches that would work. I think it makes sense to approach it from a reference-counting perspective (reference-counting each semaphore in the dictionary). Also, we want to make the decrement-count-and-remove operation atomic, so I just use a single lock (making the concurrent dictionary superfluous):
Here is a KeyedLock class that is less convenient and more error prone, but also less allocatey than Stephen Cleary’s AsyncDuplicateLock . It maintains internally a pool of SemaphoreSlim s, that can be reused by any key after they are released by the previous key. The capacity of the pool is configurable, and by default is 10.
This class is not allocation-free, because the SemaphoreSlim class allocates memory (quite a lot actually) every time the semaphore cannot be acquired synchronously because of contention.
The lock can be requested both synchronously and asynchronously, and can also be requested with cancellation and timeout. These features are provided by exploiting the existing functionality of the SemaphoreSlim class.
The implementation uses tuple deconstruction, that requires at least C# 7.
The KeyedLock class could be easily modified to become a KeyedSemaphore , that would allow more than one concurrent operations per key. It would just need a maximumConcurrencyPerKey parameter in the constructor, that would be stored and passed to the constructor of the SemaphoreSlim s.
Note: The SemaphoreSlim class when misused it throws a SemaphoreFullException . This happens when the semaphore is released more times than it has been acquired. The KeyedLock implementation of this answer behaves differently in case of misuse: it throws an InvalidOperationException("Key not found.") . This happens because when a key is released as many times as it has been acquired, the associated semaphore is removed from the dictionary. If this implementation ever throw a SemaphoreFullException , it would be an indication of a bug.
Locking asynchronous file operations in Jerry Nixon’s StorageHelper
When I was writing my first Windows 8 app store application, I was also for the first time using the async/await
feature introduced in .net 4.5.
Using async/await makes a lot of things very simple to write, easy to read later and keeps the application’s user interface (UI) responsive. But in some scenarios a re-entrance situation arises that you are not normally confronted with when doing things just one after the other.
I think file operations are a good example for this re-entrance; particularly when they are executed from within event handlers.
When handling events that get triggered from the UI, like for example a button click, it is fairly common to disable
the button as one of the first statements in the event handler and enable it again when the event handler returns, at least in asynchronous mode. This pattern works fine as long as there is no other button that will call the same code, or for that matter code that uses the same resources, like a file. (I am not talking about MVVM patterns here, so I ignore how buttons are disabled via MVVM.)
One problem with calling things asynchronously is, that while awaiting an asynchronous function call, other stuff may happen and you have little idea what that other stuff will be.
A situation I came across is this:
· A Windows app store application has two frames.
· There is no given sequence in which these frames
are called.
· Both frames need some data that comes from a
file.
· The data needs to be written back into that file
when the frames close (are navigated from).
· Both frames use the same data and therefore the
same file.
Typically, I would put the loading and saving of the data into OnNavigatedTo/ From event handlers for each frame.
In the pre- async/await times, if you don’t worry about a “little” delay for the saving and loading, putting these calls directly into the event handlers is no problem, it just slows down the UI going from one frame to the next. But if the file access takes more time, the delay becomes an issue. Also, the situation in Windows 8 app store apps is, that there are no synchronous versions of file access functions. So you are forced (by design) to do it asynchronously to keep the UI responsive, hence the event handlers become async .
The signature of the event handlers however is void and the event handlers will not be awaited by the framework. So they become -what Jerry Nixon calls- “FireAndForget” as soon as the signature is changed to async and an await is added to the function body: They return when the first await within the functions body is hit. When navigating from one frame to the other, the saving and the loading code may overlap and throw an ACCESSDENIED exception.
In the pre- async/await days you could have used a thread or task to do the file access asynchronously and keep the UI responsive. With the async/await feature this is not necessary anymore, because now you can just use the async file access functions and await them. In the pre- async/await days it would have been a common approach to put a lock around the file access to handle the situation that two threads try to execute the same code and access the same file at the same time. With async/ await locking is still an issue, but trying to put a lock around an await function call will give an error, saying that this is not allowed. Also a lock would not work in a re-entrance scenario, because the lock is still on the same thread (the UI thread in the case of event handlers).
To get around this problem I used the AsyncLock from Stephen Toub (http://blogs.msdn.com/b/pfxteam/archive/2012/02/12/10266988.aspx Building Async Coordination Primitives, Part 6: AsyncLock) to lock via a using block around any file access the StorageHelper functions perform. Please note that my contribution here is rather small.
First I bring in the two classes from Stephen Toub’s articles:
AsyncSemaphore and AsyncLock
Then in Jerry Nixon’s StorageHelper class I add a member:
private static readonly AsyncLock m_lock = new AsyncLock();
this is my only lock that I then use to synchronize all file access.
Writing a file for example:
using ( await m_lock.LockAsync())
// now Jerry’s code
var _File =
await CreateFileAsync (key,
location,
// convert to string
var _String = Serialize(value);
// save string to file
await Windows.Storage.FileIO.WriteTextAsync (_File, _String);
ret = await FileExistsAsync (key,location);
and reading a file
using ( await m_lock.LockAsync())
// now Jerry’s code
var _File = await GetIfFileExistsAsync (key,location);
return default (T);
var _String = await FileIO.ReadTextAsync (_File);
var _Result = Deserialize <T>(_String);
The using also takes care about the possibility of an exception being thrown, so the AsyncLock m_lock will be released if this should happen.
Please note that disabling parts of the UI in both frames may still be needed because – as already outlined above – the OnNavigatedTo event handler becomes “FireAndForget” when async and await are added to the definition. So as long as the fired and forgotten event handler has not done its work, the data it is supposed to read, may not be consistent.
Fun with file locking
If you are developing code that uses distributed synchronization or messaging, you sometimes might need to use files as a locking mechanism. This can be useful because files are persistent (beyond thread, process, or even power session lifetime), and access to them is synchronized between multiple processes if you select the proper file access and sharing modes.
C# example of taking a file lock:
using ( FileStream lockFile = new FileStream (
// 1. Read lock information from file
// 2. If not locked, write the lock information
If multiple threads or processes are accessing this lock file, the first process to open the file will lock it, blocking others from accessing it (more on why FileShare .Delete and not FileShare .None in a moment).
This provides the synchronization necessary for safe concurrent access of the file lock. Clients using the code should try to open the file in a try/catch loop, until a certain timeout is reached, to provide the blocking behavior for the file lock.
Where it gets a bit more interesting is when you are done with the lock, and want to release it.
A good way to do this is to delete the file (you could also write the file to indicate that you no longer hold the lock, but who wants to have left over lock files?).
Naturally, you’ll want to first open the file with the same exclusive access as you did when locking it, to insure that you are still the one that holds the lock. Unfortunately, it turns out that there isn’t a way to delete the file using the file handle that you have already opened. You have to use the File .Delete(string path) API which calls WIN32 DeleteFile() and re-opens the file in order to delete it, or re-open the file with FileOptions .DeleteOnClose (FILE_FLAG_DELETE_ON_CLOSE). This is where you get into trouble because you already have the file open under a lock to prevent other writers from taking it.
This is where our FileShare .Delete (FILE_SHARE_DELETE) comes in. By opening the file with this flag, we are prohibiting any lock taking operations from being performed, but allowing the file to be deleted by someone else.
C# example of releasing the file lock:
using ( FileStream lockFile = new FileStream (
// 1. Read lock information from file
// 2. If locked by us, delete the file:
The file will be deleted as soon as our handle to the file is closed at the end of the using <> scope.
Note that this example is meant as a mechanism for cooperative persistent file locking between multiple threads or processes. It is not meant as a way to guard against malicious or misbehaving code that wants to access the file, because anyone can break the rules while the file is not locked.
The benefit of this approach is that you can create persistent file locks that do not necessarily go away when an owning process terminates, and do not require you to keep the file exclusively locked for the duration of the lock. You can also provide lock override or timeout semantics on top of this mechanism that would not be possible with an exclusive lock approach.
You can also download the example file lock library and source code (as is, no guarantees, no limitations on use).
Using this library you can create file locks like this:
using ( FileLock l = new FileLock (path, lockId, “mylock”))
// do stuff under the file lock …
> // lock automatically released here
Of course, there are other ways to do inter-process locking on Windows, including global mutexes, that may be more appropriate depending on the situation.
Name already in use
If nothing happens, download GitHub Desktop and try again.
Launching GitHub Desktop
If nothing happens, download GitHub Desktop and try again.
Launching Xcode
If nothing happens, download Xcode and try again.
Launching Visual Studio Code
Your codespace will open once ready.
There was a problem preparing your codespace, please try again.
Latest commit
Git stats
Files
Failed to load latest commit information.
README.md
Asynchronous file lock that can auto lock and auto seek.
- async file locks (exclusive and shared)
- auto lock before doing any read or write operation
- auto seek before doing any read or write operation
- manual lock/unlock
async-file-lock should work on any platform supported by libc.
About
Asynchronous file lock that can auto lock and auto seek.
Topics
Resources
License
Stars
Watchers
Forks
Releases
Packages 0
Languages
Footer
© 2023 GitHub, Inc.
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
