Using production source maps securely in ASP.NET Core

Introduction

The goal of this article is to convince people that they should use their JavaScript and CSS source maps in production. A lot of developers are uneasy about doing this though; the main reason being security. However, I have written a piece of OWIN middleware which solves this problem, at least for ASP.NET Core applications.

I'll start by going over why having source maps in production is great for a developer. I'll then cover the risks associated with having them, then I'll talk about some bad solutions which are already used by some developers to try and mitigate those risks. Finally I'll talk about the best solution and how this OWIN middleware fits in.

Why source maps are needed in production

Back in 2015 I was assigned a production bug in an ASP.NET MVC 4 application. There was a particular page where a user would complete a form and then click a submit button. The submit button had stopped working and nothing would happen if the user clicked it.

I reproduced the issue myself and opened Chrome Developer Tools to view the console error. I saw something like this:

Uncaught ReferenceError: g is not defined 
	at a.b(app.min.js)1:200

This was because our JavaScript code was concatenated and minified in our test and production environments, using the App_Start\BundleConfig.cs file. This error message meant nothing to me without source maps. Unfortunately, whilst the bundleconfig.json file in ASP.NET Core supports JavaScript source maps, the ASP.NET MVC BundleConfig does not.

Okay, no biggie, I'll just run the application locally in my development environment and read the actual error message.

The bug didn't occur in the development environment though. Uh oh...

Okay... this must have been an issue with the BundleConfig. Sometimes there might be issues related to bundling and minification of scripts, for example if you try loading certain scripts in the wrong order.

But then I checked the test environment and the error wasn't being thrown there, despite having the exact same BundleConfig settings.

This was a bug which could only be reproduced in the production environment...and I had absolutely no idea how to find it, aside from completely disabling the BundleConfig and doing a production deployment. Ew.

The minified app.min.js file contained the source code of around 40 JavaScript source files and the bug was in one of those. At best, I could maybe narrow the number of files down to about 10. After that, the only thing left to do was to meticulously comb through the Git commits for each of those files and figure out what changed since it last worked. This was made harder by the fact that nobody in the team knew exactly how long the bug had been present for, as there were no automated UI tests for this particular feature and it was obscure enough that the bug could have been around a while before it was reported by an end-user.

This was a nightmare. This was the kind of bug which makes you question all the life choices you've made which have led to you sitting at a desk, tearing your hair out.

To tell you the truth, I can't remember exactly what the bug was caused by, or how I fixed it. What I do remember though is that it took me an entire Wednesday, Thurday and Friday to resolve the issue. I was then drunk for most of the weekend.

I also remember that when I came back into work the next week, the first thing I did was insist to my manager that I ditch the BundleConfig and replace it with a more sophisticated front-end build pipeline (i.e. Gulp, Grunt, Webpack), which could generate source maps. I also insisted that these source maps were available in all environments.

I then migrated things over to Grunt and then subsequently Gulp some time afterwards. I never had to tear my hair out like this again!

So what are the problems?

Using source maps in production is fantastic for the development team, however there are some issues which should really be addressed before doing this.

Anyone can view and steal the front-end code

Some source maps look like this:

{
	"version" : 3,
	"file": "out.js",
	"sourceRoot": "",
	"sources": ["foo.js", "bar.js"],
	"sourcesContent": [null, null],
	"names": ["src", "maps", "are", "fun"],
	"mappings": "A,AAAB;;ABCDE;"
}

The source map file itself doesn't contain the source code. For this source map to work, the files "foo.js" and "bar.js" must be paths of files which are accessible to the client, so even if someone doesn't open developer tools or download the source map file, those source files are still open to be downloaded by the public.

Source maps can also be written in this format:

{
	"version":3,
	"sources":["content/js/site.js"],
	"names: ["surprise","alert"],
	"mappings":"AACA,IAAAA,SAAA,WACAC,MAAA","file":"site.min.js",
	"sourcesContent":["// A simple function for testing JavaScript sourcemaps in production.\r\nvar surprise = function() {\r\n    alert(\"Surprise!\");\r\n}"]
}

"sourcesContent" contains the original source code for each source file named in the "sources" property. The source file paths don't have to be valid and accessible paths to the physical files, they just describe the logical structure of the files. This means that all of the source files don't have to be publically accessible, however it does mean that all someone has to do is view this source map file and they have all of your source code.

Someone could potentially copy the front-end source code and use it for a competing product, if it turns out there's some proprietary logic in the front-end that they want.

Anyone can debug or modify the front-end logic

Using the developer tools of your browser, you can set breakpoints in the original source code as well as view variable values and execute any functions you want. This is a problem as it would allow someone to clearly see the front-end logic of your application and potentially modify its behaviour.

Ideally a developer should not be relying on the integrity of the front-end code alone. Form validation for example should always be performed on the back-end, but front-end validation can also be added in order to improve performance for the end-user and reduce server load. Not all developers do this though and I've personally seen production front-end code which validates form fields, which are not subsequently validated on the server. In this situation, a malicious user could potentially bypass the form validation and submit values which should not be allowed.

In an ideal world the front-end should also not contain any crucial business logic and should instead make HTTP calls to a server-side application instead. Again, things are not always done like this though. I've seen a production AngularJS application which had most of the business logic written on the front-end. It would perform HTTP requests to an API but this would mostly be for CRUD operations and the API itself would perform minimal amounts of business logic.

It's still possible to take minified JavaScript and run it through a tool like Online JavaScript beautifier to decipher it without a source map, however it's a lot harder to understand the logic when the names of all the functions, variables etc. are minified and the comments are missing. You don't want to make it easy for someone by giving them the source map though.

Some bad solutions

If you do a search online for "hide production sourcemaps", you will find a few suggestions on how to solve these problems. Unfortunately, all of these solutions seem to either create their own problems, or are just simple over-engineered for the problem at hand.

Removing source maps from production

In December 2015 I asked a question on StackOverflow about the best way to hide my production source maps from the public. The initial response I got was that I shouldn't be using source maps in production.

This obviously isn't helpful to us, as we've established that having source maps in production is beneficial. I find it infuriating when someone asks on a forum "I have source maps in production, how can I hide them from the public?" and someone says "You shouldn't need those!" instead of addressing the actual problem.

On different forums and blogs, various other solutions have been suggested, such as running different build tasks for test and production environments and making sure source maps aren't generated in production. Another is to generate the source maps but then run another task which deletes the source map file or doesn't publish it in a production deployment. Depending on your build and deployment system, writing these additional tasks and running them at the correct stages can be messy.

Even if you do decide that having source maps in production isn't a good idea, I would say it's a bad idea to have different build and release steps for each environment. I personally do everything I possibly can to ensure my test environent mimics the production environment as much as possible, so I can be more certain that something working in the test environment is going to work in production.

Host your source maps privately, on a server only accessible to your team

A while ago this advice was given in a Sentry blog post:

Private Source Maps

Up until this point, all of our examples assume that your source maps are publicly available, and served from the same server as your executing JavaScript code. In which case, any developer can use them to obtain your original source code.

To prevent this, instead of providing a publicly-accessible sourceMappingURL, you can instead serve your source maps from a server that is only accessible to your development team. For example, a server that is only reachable from your company’s VPN.

//# sourceMappingURL: http://company.intranet/app/static/app.min.js.map

When a non-team member visits your application with developer tools open, they will attempt to download this source map but get a 404 (or 403) HTTP error, and the source map will not be applied.

Presumably in your build and deployment steps, you'd have to generate your source maps, somehow send them to your internal company server and then write the sourceMappingURL to your minified JavaScript file.

Here are some of the issues with this that I can think of off the top of my head:

  • Not everyone has on-premise servers which are only accessible through their company's network. For example, I've worked in a team which only hosted applications in Azure and sometimes it wasn't feasible or cost-effective to restrict access to just our network.

  • How do you handle granular access within your company? The server might be located internally but you don't necessarily want everyone in the company being able to access the source maps. How would you restrict it to specific departments, teams or individual employees?

  • What do you do if your build and deployment pipeline doesn't have access to your company's server? For example, you might be using a system which is hosted in the cloud, such as Visual Studio Team Services. You'd have to somehow make your server accessible to the build/release agents, which might be easier said than done if they have dynamic or unknown IP addresses. Managing this might be a lot of work.

  • How would we easily handle storing of source maps for diffrent environments? What if other products and teams need their source maps stored too? Do we need to have separate servers for those? Do we have to somehow build a system which prevents conflicts between different products and environments having the same source map file names?

  • Certain companies use some form of online error logging platform, such as Sentry, TrackJS, Errorception or Bugsnag. These tools require your source maps to be available and essentially we have the same problem as the build agents. If the source map can't be accessed unless you work in the company, you can't use the fancy deminificiation features of an error logging system. A lot of error logging products offer a service which allows you to upload your private source map to be stored on their system, via an API call. This mitigates the issue of the system not being able to access your privately hosted source map, however this is another ugly step to add to your deployment pipeline. It's definitely useful for applications which are hosted internally and have no external means of retrieving the source map, but other than that it's not the most effective solution.

Removing the sourceMappingURL comment from your minified file in production

There's a solution described in a blog post by Errorception on private source maps:

PRIVATE SOURCE MAPS

If this //#sourceMappingUrl comment is removed from your minified file, your source maps are now effectively private. This is because no one can know where you've put your files if there isn't a link pointing them to them. HTTP doesn't have any discovery mechanism built in, and a secret path is just as unguessable as a password, since no one else knows the secret. (This assumes that you don't have directory listing turned on.)

So, this is how private source maps work in Errorception: You specify a secret folder name in Errorception's Settings > Source Maps. This secret folder should be as unguessable as you would want a password to be. Then, modify your build/deploy script such that Errorception can find your source map on your web server by constructing a path that incorporates this secret folder. (More about this below.) Once the crawler gets your source map file, it has everything it needs to figure its way about your code.So, this is how private source maps work in Errorception: You specify a secret folder name in Errorception's Settings > Source Maps. This secret folder should be as unguessable as you would want a password to be. Then, modify your build/deploy script such that Errorception can find your source map on your web server by constructing a path that incorporates this secret folder. (More about this below.) Once the crawler gets your source map file, it has everything it needs to figure its way about your code.

This idea also has its obvious issues:

  • Developers can't view or debug the source code in production, because web browsers won't be able to find the source maps as the sourceMappingURL is missing. The best they can do is use Errorception to find the cause of an error that was logged. What if the error isn't logged though? What's your recourse then?

  • You're still publically exposing your source code. Granted, there isn't a discovery mechanism and people won't necessarily know where they are but the point is that you shouldn't be publishing your source files publically at all.

  • This is another solution which requires changing your build and deployment process. Again, this is something we want to avoid as much as possible, especially if the location of these files is subject to change.

The best solution...is to just use auth!

Let's just go back to basics and stop thinking about source maps for a moment. In the most simple terms, what are we actually trying to achieve here?

We're trying to restrict a specific set of resources to a specific set of users. That's it. It really isn't any more complicated than that.

Source map files in this case are a type of resource we want to protect from everyone who doesn't meet certain criteria. This is exactly the same problem as having a specific page or feature which you want to restrict to certain users. This is a perfect use case for authentication and authorisation.

Introducing the SourceMapSecurity middleware for ASP.NET Core

I built SourceMapSecurity, my own OWIN middleware for ASP.NET Core which allows you to set your own authentication/authorisation rules for allowing or disallowing requests for source map files.

It's available on NuGet and the GitHub page has documentation which details the prerequisities and how to add the middleware to your ASP.NET Core application.

How it works for DavidOmid.com

My website has source maps for the JavaScript and CSS, which are published under the wwwroot folder. However, you can't access them unless you're logged in. I'm using Azure Active Directory for authentication and the only person who has access is myself.

This is how I've implemented the SourceMapSecurity middleware in the Startup.cs class of my project:

app.UseSourceMapSecurity(new SourceMapSecurityOptions()
{
	IsAllowedAsync = async (context) =>
	{
		if (!env.IsDevelopment() && !context.User.Identity.IsAuthenticated)
		{
			return false;
		}
		return true;
	}
});

What's going on here?

If an HTTP request is made for a source map file, the current environment is checked. If the application is running in the development environment, it's fine for the source maps to be returned. If, however, it's running in a different environment (i.e. Test, Production), it's only fine to return the source map if the client is logged in.

How does this work?

  • app.UseSourceMapSecurity adds the SourceMapSecurity middleware to my request pipeline. Providing an instance of SourceMapSecurityOptions allows you to define the specific rules regarding who has access and who doesn't.
  • IsAllowedAsync is the method which will be executed whenever a client attempts to access a source map file. It returns a boolean, indicating whether or not the client should be allowed or disallowed access to the source map. SourceMapSecurityOptions allows you to provide an implementation of this method. If you don't, by default all requests for source maps will be denied, for all clients.

Test it yourself!

These are the locations of my source map files:

https://www.davidomid.com/js/lib.min.js.map

https://www.davidomid.com/js/site.min.js.map

https://www.davidomid.com/css/lib.min.css.map

https://www.davidomid.com/css/site.min.css.map

You can't access them, but I can:

A hidden source map in production

Why is this the best solution?

This solution provides more extensibility. In a more complicated application with multiple users, you'd probably check the user's claims in detail to determine which permissions they should have. For example, a user could have a role which is specifically allowed access to source maps, which other users might not have.

I recommend using a full-featured identity provider for authentication such as Azure Active Directory, however this might not always be feasible.

Some online error logging tools such as Sentry do provide the option of accessing your web application using secure tokens in the request headers:

Secure Access to Source Maps

If you want to keep your source maps secret and choose not to upload your source maps directly to Sentry, you can enable the “Security Token” option in your project settings. This will cause outbound requests from Sentry’s servers to URLs originating from your “Allowed Domains” to have the HTTP header “X-Sentry-Token: {token}” appended, where {token} is a secure value you define. You can then configure your web server to allow access to your source maps when this header/token pair is present. You can alternatively override the default header name (X-Sentry-Token) and use HTTP Basic Authentication, e.g. by passing “Authorization: Basic {encoded_password}”.

Some other tools don't support this feature but usually provide their IP addresses of their servers which make outbound connections to your web applications. You can then whitelist them and treat it as a basic form of authentication.

The SourceMapSecurity middleware works well for IP addresses too. You can essentially use it for whichever form of authentication and authorisation you're comfortable using. Here's an example I quickly wrote which whitelists TrackJS's IP addresses.

// This hardcoded list is just for demonstrating a point. 
// You're better off using a database or config ile for storing IP addresses. 
HashSet<string> allowedClientIpAddresses = new HashSet<string>
{
	// TrackJS servers
	"142.4.218.95",
	"167.114.172.73",
	"198.27.94.180"
};
app.UseSourceMapSecurity(new SourceMapSecurityOptions()
{
	IsAllowedAsync = async (context) =>
	{
		if (!allowedClientIpAddresses.Contains(context.Connection.RemoteIpAddress.ToString()))
		{
			return false;
		}
		return true;
	}
});

Simple, right?

Conclusion

I believe source maps should be used in production and should be locked down with authentication and possibly further authorisation. At the bare minimum, IP whitelisting should be used.

I also believe that ideally all online error logging products out there should have some mechanism of connecting to web applications using auth tokens. This seems to me like a good candidate for using standards such as OAuth2. For example, developers could grant a product access to the source maps, but then revoke that permission at a later stage if necessary. Even if they all just generated a GUID for each web application and sent that in an HTTP request header like Sentry, I feel like that would work quite well as an authentication mechanism. It also wouldn't matter what the IP addresses of the servers are.

IP whitelisting does work pretty well though, even as the most basic form of authentication, and I firmly believe that any authentication is the best solution here.

David Omid

Hi, my name's David and I'm a senior software engineer from London.