We recently used a symbolic link to an Azure file share to transparently replace a local folder in which data sets are cached for download. That means that the existing service transparently copies the data sets to an Azure file share without having to change anything in the code to do so. With a small adaptation of the code, we can now provide download links to data in the Azure file share so this process is also transparent for the clients downloading the data sets.
You can already guess the reason for this exercise. We did this to fix a bandwidth issue on-premises by creating an easy workaround with minimal code changes. As more and more clients download more and more data sets, this service consumes too much bandwidth. This means we have to throttle the service and/or implement QoS to it. While this helps the other services using that internet connection, it does nothing to improve download speeds for the clients. This is just an example and is not meant as architectural or design advice. It is an interim fix to an existing problem. This trick is something that is used with AKS as well for example.
How to add a symbolic link to an Azure file share
CREATE AN AZURE FILE SHARE
Create a storage account and create a file share.
Dealing with the credentials needed for this is easy. All we need to do is add the information into the credential manager as a Windows credential. That would be the user, the password, and the file share UNC path. Note that here the password is our storage account key.
Grab the info you need from the “connect” settings for your Azure file share. We will not map the the files hare to a drive, so there is no need to run this PowerShell script.
So in this example that is:
Internet or networkk address: \\datasets.file.core.windows.net\fscache
User name: localhost\datasets
We will add these credentials to the Credential Manager as Windows Credentials.
That is it, if you entered everything correctly, this will work.
CREATING THE SYMBOLIC LINK
Once you have added the credentials creating the symbolic link is very easy.
You do need to take care you create the symbolic in the right place in your folder structure. But other than that, that is all you need to do.
The symbolic link is available and can be used transparently by the service/application.
To test the file share in Azure you can upload or download data via azcopy or Azure Storage Explorer. The download functionality in our case is handled in the code, But here is a quick example of how to do a download it via azcopy using a shared access key signature.
Mind you, this was the easy and quick fix for a problem this service was facing. This is not a design or architecture. We are considering replacing the symbolic link solution with Azure File Sync. With a bandwidth cap and QoS on-premises, we would offer the primary download link to the cloud. There they can get all the bandwidth Azure can offer. Next to that, we would have an alternative link, marked as slow, that still points to the on-prem version of the data. This means the current implementation is still fully functional even when the Azure files share has an issue. Sure, the local copy comes with a significantly reduced performance, but it provides a failsafe.
Well, the future lies in turning this into a solution running 100% in the cloud. Now, due to a large number of dependencies on various on-premies data sources, this is a long-term effort. We decided no to let perfection be the enemy of the good and fixed their biggest pain point today.
For sure, the use of a symbolic link to access an Azure file share is not something that will amaze people that have been working in the cloud for a while. It is however a nice example of how the use of Azure combined with on-premises services can result in a hybrid solution that solves real-world problems
This particular scenario enables them to distribute their data sets without having to worry about bandwidth limitations on-premises. That means they do to invest in a bigger internet pipe and a firewall with more throughput, or having to port their service and all its dependencies to a full-blown Azure solution.
Sometimes successful and cost-effective solutions come in the form of little tweaks that allow us to fix pain points easily.
About the Author:
My name is Didier Van Hoye, better known as WorkingHardInIT on twitter. I’m IT pro that has been at “IT” for over 20 years. I’m convinced you cannot design what you cannot build and vice versa. So while I’m an “Architect” I’m still in the trenches with my boys and gals. Empty suits or hollow bunnies are neither wanted nor needed. In IT you live by the sword and you die by the sword. There is no hiding when you mess up, all our mistakes are in plain sight of everyone using what we build.
I have what some people like to refer to as “alphabet soup” that I’m allowed to use since I earned the titles those acronyms represent. I’m an MCP, MCSD, MCDBA, MCSA, MCSE & MCTP. I’ve been presented with the Microsoft Most Valuable Professional (MVP) award for Hyper-V. I’m also Microsoft Extend Experts Team member in Belgium, a DELL TechCenter Rockstar and last but certainly not least a VEEAM Vanguard.. All programs driven by and for the community.
Hoye Can, D. (2021). Symbolic Link to an Azure File Share. Available at: https://blog.workinghardinit.work/2021/06/24/symbolic-link-to-an-azure-file-share/ [Accessed: 8th July 2021].