AngularJs native multi-file upload with progress

Problem

If you are reading this post you probably agree with me that AngularJs is pretty awesome. Because it’s so new, there are a number of gaps in the framework. My last post was inspired by filling a gap, and now again, I will present a directive and service I wrote because it doesn’t exist within the framework.

I am working on an application that requires users to upload files. As it’s 2013 and we are living in an asynchronous world, I wanted an intuitive UI where my users can upload files, along with some data and view upload progress. I’ve found a bunch of stuff that wraps jquery implementations, but nothing native, so, I guess I’ll build one.

The solution consists of a service and directive. The code presented in this post can be found on github

• Upload service unit test
• The demo requires Node, once you’ve installed it you can run npm install express to run the demo server.

Design Goals

1. Native – no dependency on external frameworks
2. Give the parent scope a reasonable API for managing file uploads
3. Testable

Supported Browsers

This solution relies on the XMLHttpRequest2 object which limits it’s browser support. I don’t know of any polyfills for XmlHttpRequest2 off the top, but if one exists please let me know in comments.

• IE 10
• FF 4
• Chrome 7
• Safari 5
• Opera 12

View detailed support information on Can I use

Implementation

As usual, I started “outside-in”, that is, I had an idea of the API I wanted to work with from the parent scope (i.e. the angular module that houses the directive) and then figured out how to build it. The directive is an element that is replaced in the DOM with the appropriate html elements.

Events

These are specified as directive attributes and refer to methods on the parent scope.

onProgress(percentComplete)

Fires as the browser is uploading data

onDone(files, responseData)

Fires when the upload is complete responseData is the data returned by the server

onError(files, type, msg)

Fires when the browser encounters an error

getAdditionalData()

This method is called before the upload, and it should return a json object containing data that is to be posted to the server along with the files

Properties

These are also specifed as directive attributes and can be any valid angular expression. They will be evaluated in the parent’s scope before being used by the directive.

chooseFileButtonText

The text displayed on the button that launches the file picker

uploadUrl

The URL that will process the posted data

maxFiles

The maximum number of files to post. If the user selects more files than indicated an error of type TOO_MANY_FILES is raised

maxFileSizeMb

The maximum size per file allowed to be uploaded. If a file larger than the specified maximum is selected, the entire upload operation is aborted and an error of type MAX_SIZE_EXCEEDED is raised

autoUpload

A boolean value indicating whether the upload should start as soon as the user selects files. If this value is falsey a button is shown, which a user must click in order to upload data

uploadFileButtonText

The text displayed on the button that uploads data (only seen if autoUpload is falsey)

File upload directive

The directive is concerned with manipulating and managing the DOM on behalf of it’s parent. It is not responsible for uploading the actual files. This separation of concerns is important for a couple of reasons. First and foremost it allows us to test the upload functionality, separately from the directive. Also, it would make the directive code file big and unwieldy. Directive definitions are a bit cumbersome – there contain a mix of nested objects, closures, string literals, arrays and functions. Adding more complex functionality directly to the directive definition object would be ugly.

angular
	.module("lvl.directives.fileupload", ['lvl.services'])
	.directive('lvlFileUpload', ['uuid', 'fileUploader', function(uuid, fileUploader) {
		return {
			restrict: 'E',
			replace: true,
			scope: {
				chooseFileButtonText: '@',
				uploadFileButtonText: '@',
				uploadUrl: '@',
				maxFiles: '@',
				maxFileSizeMb: '@',
				autoUpload: '@',
				getAdditionalData: '&',
				onProgress: '&',
				onDone: '&',
				onError: '&'
			},
			template: '<span>' + 
						'<input type="file" style="opacity:0" />' +
						'<label class="lvl-choose-button" ng-click="choose()">{{chooseFileButtonText}}</label>' +
						'<button class="lvl-upload-button" ng-show="showUploadButton" ng-click="upload()">{{uploadFileButtonText}}</button>' +
					  '</span>',
			compile: function compile(tElement, tAttrs, transclude) {
				var fileInput = angular.element(tElement.children()[0]);
				var fileLabel = angular.element(tElement.children()[1]);

				if (!tAttrs.maxFiles) {
					tAttrs.maxFiles = 1;
					fileInput.removeAttr("multiple")
				} else {
					fileInput.attr("multiple", "multiple");
				}

				if (!tAttrs.maxFileSizeMb) {
					tAttrs.maxFileSizeMb = 50;
				}

				var fileId = uuid.new();
				fileInput.attr("id", fileId);
				fileLabel.attr("for", fileId);

				return function postLink(scope, el, attrs, ctl) {
					scope.files = [];
					scope.showUploadButton = false;

					el.bind('change', function(e) {
						if (!e.target.files.length) return;

						scope.files = [];
						var tooBig = [];
						if (e.target.files.length > scope.maxFiles) {
							raiseError(e.target.files, 'TOO_MANY_FILES', "Cannot upload " + e.target.files.length + " files, maxium allowed is " + scope.maxFiles);
							return;
						}

						for (var i = 0; i < scope.maxFiles; i++) {
							if (i >= e.target.files.length) break;

							var file = e.target.files[i];
							scope.files.push(file);

							if (file.size > scope.maxFileSizeMb * 1048576) {
								tooBig.push(file);
							}
						}

						if (tooBig.length > 0) {
							raiseError(tooBig, 'MAX_SIZE_EXCEEDED', "Files are larger than the specified max (" + scope.maxFileSizeMb + "MB)");
							return;
						}

						if (scope.autoUpload && scope.autoUpload.toLowerCase() == 'true') {
							scope.upload();
						} else {
							scope.$apply(function() {
								scope.showUploadButton = true;
							})
						}
					});

					scope.upload = function() {
						var data = null;
						if (scope.getAdditionalData) {
							data = scope.getAdditionalData();
						}

						if (angular.version.major <= 1 && angular.version.minor < 2 ) {
							//older versions of angular's q-service don't have a notify callback
							//pass the onProgress callback into the service
							fileUploader
								.post(scope.files, data, function(complete) { scope.onProgress({percentDone: complete}); })
								.to(scope.uploadUrl)
								.then(function(ret) {
									scope.onDone({files: ret.files, data: ret.data});
								}, function(error) {
									scope.onError({files: scope.files, type: 'UPLOAD_ERROR', msg: error});
								})
						} else {
							fileUploader
								.post(scope.files, data)
								.to(scope.uploadUrl)
								.then(function(ret) {
									scope.onDone({files: ret.files, data: ret.data});
								}, function(error) {
									scope.onError({files: scope.files, type: 'UPLOAD_ERROR', msg: error});
								},  function(progress) {
									scope.onProgress({percentDone: progress});
								});
						}

						resetFileInput();
					};

					function raiseError(files, type, msg) {
						scope.onError({files: files, type: type, msg: msg});
						resetFileInput();
					}

					function resetFileInput() {
						var parent = fileInput.parent();

						fileInput.remove();
						var input = document.createElement("input");
						var attr = document.createAttribute("type");
						attr.nodeValue = "file";
						input.setAttributeNode(attr);

						var inputId = uuid.new();
						attr = document.createAttribute("id");
						attr.nodeValue = inputId;
						input.setAttributeNode(attr);

						attr = document.createAttribute("style");
						attr.nodeValue = "opacity: 0;display:inline;width:0";
						input.setAttributeNode(attr);

						if (scope.maxFiles > 1) {
							attr = document.createAttribute("multiple");
							attr.nodeValue = "multiple";
							input.setAttributeNode(attr);
						}

						fileLabel.after(input);
						fileLabel.attr("for", inputId);

						fileInput = angular.element(input);
					}
				}
			}
		}
	}]);

Line 2: Reference the services used by this directive (included in files lvl-uuid.js and lvl-xhr-post.js).

Line 3: Inject the UUID and XHRPost services.

Line 7 - 18: Setup the directives scope (API).

Line 19 - 23: The HTML template. Note that this directive will replace the mark-up entered into the HTML file.

Line 28 - 33: Determine whether or not this instance of the directive needs to support multiple files. If it does, it adds the multiple attribute to the file input element of the template, otherwise the attribute is removed. If this instance is only meant to support a single file, don’t let the user select more than one (see the pit of success).

Line 35 - 37: Ensure there is a max file size set (default is 50MB).

Line 39 -41: Ensure the label element refers to the file input element. We do this so we can hide the file input element by setting it’s opacity to 0, and when a user clicks the label the file input is opened. This was the only way I could find to hide the stock file chooser button. If there is a way to do it with a button, please let me know in the comments or by submitting a pull request.

Line 43: The link function, where the magic happens!

Line 47: The change DOM event is triggered when files are selected by the end user.

Line 48 - 71: Validate the selected files. No-op if the user cancels, otherwise raise our onError event if any of the validation fails.

Line 73 - 79: If autoUpload is truthy then we upload the file, otherwise show the upload button.

Line 83 - 86: Collect any additional data to post to the server from the parent.

Line 88 - 110: The XHRPost service returns a deferred object from the $q service, which introduced new and relevant functionality in v1.2.0rc1 – specifically the notify method.

notify(value) – provides updates on the status of the promises execution. This may be called multiple times before the promise is either resolved or rejected.

In other words, this is the method used to let the parent know that upload progress has been made. Let’s take a closer look.

Line 91 - 98: Older versions of angular call the XHRPost service by passing in the onProgress event as a method parameter. This compensates for the fact that there is no $q.defer().notify() method.

Line 100 - 109: Newer versions rely on the $q.defer().notify() method.

Line 115 - 118: Raises an onError event and resets the file input.

Line 120 - 148: By default a file can’t be added to a file input element multiple times, and files, once selected can’t be deselected. This is problematic in the case where autoUpload is falsey, and if a validation error occurs. This method gets around this limitation by removing the input file element that has files attached to it, and replacing it with a new one. The code is straight javascript and pretty self explanatory.

Styling the buttons

Styling is easy as there are only 2 elements visible to end-users.

lvl-choose-button

This is actually a label element that triggers the file input element. It acts like a button, so I named it as such

lvl-upload-button

This is a real button element, displayed when autoUpload is falsey.

XHRPost service

As I said earlier, the post service is responsible for sending data to the server and reporting upload events to it’s consumer. I like ‘fluent’ APIs, so I wrote a mini-one for the service. If you are using the directive, you don’t need to concern yourself with this service, but it exists and can be used without the directive if you are so inclined.

post(files, data, progressCb)

• files the files to be uploaded.
• data optional, json data to be posted to the server.
• progressCb optional, if your version of the $q service doesn’t support the notify() method, pass the progress callback in as a method argument.

This method returns an object with a single method that does the actual work.

to(uploadUrl)

this method posts everything to the url specified

You can see the usages above. To review, Line 91 demonstrates how to call the post method with the progressCb parameter, and Line 100 demonstrates how to use the notify method when it’s available. NOTE If you don’t pass in the progressCb parameter AND the notify method is not available you will not receive progress notifications.

var module;

try {
    module = angular.module('lvl.services');  
} catch (e) {
    module  = angular.module('lvl.services', []);
}

module.factory('fileUploader', ['$rootScope', '$q', function($rootScope, $q) {
	var svc = {
		post: function(files, data, progressCb) {

			return {
				to: function(uploadUrl)
				{
					var deferred = $q.defer()
					if (!files || !files.length) {
						deferred.reject("No files to upload");
						return;
					}

					var xhr = new XMLHttpRequest();
					xhr.upload.onprogress = function(e) {
						$rootScope.$apply (function() {
							var percentCompleted;
						    if (e.lengthComputable) {
						        percentCompleted = Math.round(e.loaded / e.total * 100);
						        if (progressCb) {
						        	progressCb(percentCompleted);
						        } else if (deferred.notify) {
							        deferred.notify(percentCompleted);
							    }
						    }
						});
					};

					xhr.onload = function(e) {
						$rootScope.$apply (function() {
							var ret = {
								files: files,
								data: angular.fromJson(xhr.responseText)
							};
							deferred.resolve(ret);
						})
					};

					xhr.upload.onerror = function(e) {
						var msg = xhr.responseText ? xhr.responseText : "An unknown error occurred posting to '" + uploadUrl + "'";
						$rootScope.$apply (function() {
							deferred.reject(msg);
						});
					}

					var formData = new FormData();

					if (data) {
						Object.keys(data).forEach(function(key) {
							formData.append(key, data[key]);
						});
					}

					for (var idx = 0; idx < files.length; idx++) {
						formData.append(files[idx].name, files[idx]);
					}

					xhr.open("POST", uploadUrl);
					xhr.send(formData);

					return deferred.promise;				
				}
			};
		}
	};

	return svc;
}]);

Line 1 - 8: I thought this was a really clever way to keep all of my services in a single namespace. When calling angular.module(namespace) angular tries to find the module in it’s internal list of loaded modules, if it doesn’t exist it throws an error. Calling angular.module(namespace, [dependent on namespaces]) creates the module in the current angular context. So, I was really psyched when I thought of this, but now, I think there is a shortcoming – all of the services/directives in the namespace need to depend on the same namespaces. I’ve not decided if this is a deal breaker or not, so I left it in. If you have ideas about this let me know in the comments.

Line 9 - 10: Setup the factory with the dependencies it needs – specifically rootScope so it can cause a digest cycle when upload events fire, and $q service so we can return promises.

Line 11: This function simply captures the parameters in a closure, and returns an object with the to function.

Line 14 - 75: This function does all the heavy lifting.

Line 16 - 20: Setup the promise object, and ensure there are files to upload. If no files are present, reject the promise and return.

Line 22 - 52: This creates the XMLHttpRequest object resolves/rejects/notifies on the promise object as appropriate. The only trickiness is on Line 28 - 30 where the service determines if it should use the callback or the promise’s notfiy method.

Line 54 - 64: Populate the data to send to the server. Different frameworks will handle the posted data differently. I’ve included a simple NodeJs in the repo to give you a sense, but server implementation details will vary. Feel free to send pull requests with server samples, or leave sample server code as comments.

Line 66 - 67: Post the data!

Line 70: Return a promise to the service’s consumer.

Testing the service

This project was complex enough to get me off my ass and start writing client-side unit tests. So, I am taking baby-steps here, and I started with Jasmine which is, I believe, the angular team’s testing framework of choice (along with the Karma test runner, which I didn’t use).

There were 2 hurdles involved in writing the unit tests

1. Mocking service dependencies
2. Mocking the XMLHttpRequest object

Mocking service dependencies

I spent a surprising amount of time on this, and it ended up being quite easy. The trick that I found out way too late is to include the file angular-mock.js after your angular library in the test file. This allows you to create testable modules. Once you know this trick, it’s easy to mock the dependencies with Jasmine.

Mocking the XMLHttpRequest object

Another thing I spent a lot of time on. I investigated sinon.js, which may be a nice library, but proved to be overly complicated for my needs. I ended up using Jasmine.Ajax with some minor modifications.

All of the test code is available in the repository. I don’t want to dilute this post with testing details, but I’m happy to answer questions about the tests in comments.

Using the directive

We’re done — easy peasy lemon squeezy. The repo contains an integration test which posts files to the included NodeJs server, but here’s the high level.

Html

<script src="../script/lvl-uuid.js"></script>
<script src="../script/lvl-xhr-post.js"></script>
<script src="../script/lvl-file-upload.js"></script>
<style>
   .lvl-choose-button {
      //whatever
    }

   .lvl-upload-button {
     //whatever
   }
</style>

<lvl-file-upload
   auto-upload='false'
   choose-file-button-text='Choose files'
   upload-file-button-text='Upload files' 
   upload-url='http://localhost:3000/files' 
   max-files='10'
   max-file-size-mb='5'
   get-additional-data='getData(files)'
   on-done='done(files, data)'
   on-progress='progress(percentDone)' 
   on-error='error(files, type, msg)'/>

Handle events in your controller

Script

angular
  .module('app', ['lvl.directives.fileupload'])
  .controller('ctl', ['$scope', function($scope) {
      $scope.progress = function(percentDone) {
            console.log("progress: " + percentDone + "%");
      };

      $scope.done = function(files, data) {
            console.log("upload complete");
            console.log("data: " + JSON.stringify(data));
            writeFiles(files);
      };

      $scope.getData = function(files) { 
            //this data will be sent to the server with the files
            return {msg: "from the client", date: new Date()};
      };

      $scope.error = function(files, type, msg) {
            console.log("Upload error: " + msg);
            console.log("Error type:" + type);
            writeFiles(files);
      }

      function writeFiles(files) 
      {
            console.log('Files')
            for (var i = 0; i < files.length; i++) {
                  console.log('\t' + files[i].name);
            }
      }
}]);

The end

I hope you found this post useful. As always I welcome questions, comments, observations or suggestions

~~~jason

• angularjs
• directives
• javascript
• level llc
• services