If you've ever uploaded a large video file, then you know this feeling: you're 90% done, and accidentally refresh the page—having to start all over again.

In this tutorial, I'll demonstrate how to make a video uploader for your site that can resume an interrupted upload and generate a thumbnail upon completion.

Intro

To make this uploader resumable, the server needs to keep track of how much a file has already been uploaded and be able to continue from where it left off. To accomplish this task, we will give full control to the Node.js server to request specific blocks of data, and the HTML form will pick up these requests and send the necessary information to the server.

To handle this communication, we'll use Socket.io. If you've never heard of Socket.io, it is a framework for real-time communication between Node.js and an HTML web page—we'll dig more into this shortly.

This is the basic concept; we will start with the HTML form.

1. Create the HTML

I am going to keep the HTML fairly simple; all we need is an input to choose a file, a text box for the name, and a button to begin the upload. Here's the necessary code:

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta http-equiv="X-UA-Compatible" content="IE=edge" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Resumable Video Uploader</title>
  </head>
  <body>
    <div id="UploadBox">
      <h2>Video Uploader</h2>
      <span id="UploadArea">
        <label for="FileBox">Choose A File: </label
        ><input type="file" id="FileBox" /><br />
        <label for="NameBox">Name: </label
        ><input type="text" id="NameBox" /><br />
        <button type="button" id="UploadButton" class="Button">Upload</button>
      </span>
    </div>
  </body>
  <script type="module">
      // put JavaScript code here
  </script>
</html>

Notice that I have wrapped the contents in a span; we will use this later to update the page's layout with JavaScript. I'm not going to cover the CSS in this tutorial, but you can download the source code if you'd like to use mine.

2. Add an Upload Button With the FileReader API

In this tutorial, we will be using the HTML5 FileReader API, which is almost universally supported.

The FileReader class allows us to open and read parts of a file and pass the data as a Binary string to the server. Here is the JavaScript for the feature detection:

1	document.getElementById('UploadButton').addEventListener('click', StartUpload);
2	document.getElementById('FileBox').addEventListener('change', FileChosen);

The code above additionally adds event handlers to the button and file input in the form. The FileChosen function simply sets a global variable with the file—so that we can access it later—and fills in the name field, so that the user has a reference point when naming the file. Here is the FileChosen function:

let SelectedFile;
function FileChosen(evnt) {
  SelectedFile = evnt.target.files[0];
  document.getElementById("NameBox").value = SelectedFile.name;
}

Before we write the StartUpload function, we have to set up the Node.js server with Socket.io; let's take care of that now.

3. Create the Socket.io Server

As I mentioned earlier, I'll be using Socket.io for communication between the server and the HTML file. To download Socket.io, type npm install socket.io into a Terminal window (assuming that you've installed Node.js), once you have navigated to this project's directory.

The way Socket.io works is: either the server or the client "emits" an event, and then the other side will pick up this event in the form of a function with the option of passing JSON data back and forth. This is done through the WebSocket protocol, which is a method of establishing an extremely fast two-way connection to a client and server. A few other alternatives to using WebSockets for this are multipart HTTP uploads and WebRTC. However, we are only using WebSockets in this tutorial.

To get started, create an empty JavaScript file, and place the following code within it.

const app = require("http").createServer(handler),
  { Server } = require("socket.io"),
  fs = require("fs"),
  exec = require("child_process").exec,
  util = require("util");
const io = new Server(app);
app.listen(8080);

function handler(req, res) {
  fs.readFile(__dirname + "/index.html", function (err, data) {
    if (err) {
      res.writeHead(500);
      return res.end("Error loading index.html");
    }
    res.writeHead(200);
    res.end(data);
  });
}

io.on("connection", function (socket) {
  //Events will go here
});

The first five lines include the required libraries, the next line instructs the server to listen on port 8080, and the handler function simply passes the contents of our HTML file to the user when they access the site.

The last two lines are the Socket.io handler and will be called when someone connects via Socket.io.

Now, we can go back to the HTML file and define some Socket.io events.

4. Emit Some Socket.io Events

To begin using Socket.io in our page, we first need to import its JavaScript library. We will do this using ESM, a new module specification in browsers. To import Socket.io with ESM, we will use this at the start of the script we are running:

1	import { io } from "https://cdn.socket.io/4.4.1/socket.io.esm.min.js";

If you are using a bundler, you can also install socket.io-client and import it from there. However, we will not be doing that in this tutorial.

Now, we can write the StartUpload function that we connected to our button:

let FReader;
let Name;
function StartUpload() {
  if (document.getElementById("FileBox").value != "") {
    FReader = new FileReader();
    Name = document.getElementById("NameBox").value;
    let Content =
      "<span id='NameArea'>Uploading " +
      SelectedFile.name +
      " as " +
      Name +
      "</span>";
    Content +=
      '<div id="ProgressContainer"><div id="ProgressBar"></div></div><span id="percent">0%</span>';
    Content +=
      "<span id='Uploaded'> - <span id='MB'>0</span>/" +
      Math.round(SelectedFile.size / 1048576) +
      "MB</span>";
    document.getElementById("UploadArea").innerHTML = Content;
    FReader.onload = function (evnt) {
      socket.emit("Upload", { Name: Name, Data: evnt.target.result });
    };
    socket.emit("Start", { Name: Name, Size: SelectedFile.size });
  } else {
    alert("Please Select A File");
  }
}

The first line connects to the Socket.io server; next, we've created two variables for the File Reader and the name of the file, as we are going to need global access to these. Inside the function, we first ensure that the user selected a file, and, if they did, we create the FileReader and update the DOM with a nice progress bar.

The FileReader's onload method is called every time it reads some data; all we need to do is emit an Upload event and send the data to the server. Finally, we emit a Start event, passing in the file's name and size to the Node.js server.

Now, let's return to the Node.js file and implement handlers for these two events.

5. Handle the Events on the Server

The socket.io events go inside the handler that we have on the last line of our Node.js file. The first event that we'll implement is the Start event, which is triggered when the user clicks the Upload button.

I mentioned earlier that the server should be in control of which data it wants to receive next; this will allow it to continue from a previous upload that was incomplete. It does this by first determining whether there was a file by this name that didn't finish uploading, and, if so, it will continue from where it left off; otherwise, it will start at the beginning. We'll pass this data in half-megabyte increments, which comes out to 524288 bytes.

In order to keep track of different uploads happening at the same time, we need to add a variable to store everything. To the top of your file, add let Files = {};'. Here's the code for the Start event:

socket.on("Start", function (data) {
  //data contains the variables that we passed through in the html file
  const Name = data["Name"];
  Files[Name] = {
    //Create a new Entry in The Files Variable
    FileSize: data["Size"],
    Data: "",
    Downloaded: 0,
  };
  let Place = 0;
  try {
    const Stat = fs.statSync("Temp/" + Name);
    if (Stat.isFile()) {
      Files[Name]["Downloaded"] = Stat.size;
      Place = Stat.size / 524288;
    }
  } catch (er) {} //It's a New File
  fs.open("Temp/" + Name, "a", 0755, function (err, fd) {
    if (err) {
      console.log(err);
    } else {
      Files[Name]["Handler"] = fd; //We store the file handler so we can write to it later
      socket.emit("MoreData", { Place: Place, Percent: 0 });
    }
  });
});

First, we add the new file to the Files array, with the size, data, and number of bytes downloaded so far. The Place variable stores where in the file we are up to—it defaults to 0, which is the beginning. We then check if the file already exists (i.e. it was in the middle and stopped), and update the variables accordingly. Whether it's a new upload or not, we now open the file for writing to the Temp/ folder and emit the MoreData event to request the next section of data from the HTML file.

Now, we need to add the Upload event, which, if you remember, is called every time a new block of data is read. Here is the function:

socket.on("Upload", function (data) {
  var Name = data["Name"];
  Files[Name]["Downloaded"] += data["Data"].length;
  Files[Name]["Data"] += data["Data"];
  if (Files[Name]["Downloaded"] == Files[Name]["FileSize"]) {
    //If File is Fully Uploaded
    fs.write(
      Files[Name]["Handler"],
      Files[Name]["Data"],
      null,
      "Binary",
      function (err, Writen) {
        //Get Thumbnail Here
      }
    );
  } else if (Files[Name]["Data"].length > 10485760) {
    //If the Data Buffer reaches 10MB
    fs.write(
      Files[Name]["Handler"],
      Files[Name]["Data"],
      null,
      "Binary",
      function (err, Writen) {
        Files[Name]["Data"] = ""; //Reset The Buffer
        let Place = Files[Name]["Downloaded"] / 524288;
        let Percent =
          (Files[Name]["Downloaded"] / Files[Name]["FileSize"]) * 100;
        socket.emit("MoreData", { Place: Place, Percent: Percent });
      }
    );
  } else {
    let Place = Files[Name]["Downloaded"] / 524288;
    let Percent = (Files[Name]["Downloaded"] / Files[Name]["FileSize"]) * 100;
    socket.emit("MoreData", { Place: Place, Percent: Percent });
  }
});

The first two lines of this code update the buffer with the new data and update the "total bytes downloaded" variable. We have to store the data in a buffer and save it in increments, so that it doesn't crash the server due to memory overload; every ten megabytes, we will save and clear the buffer.

The first if statement determines if the file is completely uploaded, the second checks if the buffer has reached 10 MB, and, finally, we request MoreData, passing in the percent done and the next block of data to fetch.

Now, we can go back to the HTML file to implement the MoreData event and update the progress.

6. Keep Track of the Progress

I created a function to update the progress bar and the number of MB uploaded on the page. In addition to that, the More Data event reads the block of data that the server requested and passes it on to the server.

To split the file into blocks, we use the File API's Slice command. Since the File API is still in development, we need to use webkitSlice and mozSlice for Webkit and Mozilla browsers, respectively.

With this final function, the uploader is completed! All we have left to do is move the completed file out of the Temp/ folder and generate the thumbnail.

7. Create the Thumbnail

Before we generate the thumbnail, we need to move the file out of the temporary folder. We can do this by using file streams and the pump method. The pump method takes in a read and write stream and buffers the data across. You should add this code where I wrote "Generate Thumbnail here" in the Upload event:

let inp = fs.createReadStream("Temp/" + Name);
let out = fs.createWriteStream("Video/" + Name);
util.pump(inp, out, function () {
    fs.unlink("Temp/" + Name, function () {
        //This Deletes The Temporary File
        //Moving File Completed
    });
});

We've added the unlink command; this will delete the temporary file after we finish copying it. Now onto the thumbnail: we'll use ffmpeg to generate the thumbnails because it can handle multiple formats and is a cinch to install. At the time of this writing, there aren't any good ffmpeg modules, so we'll use the exec command, which allows us to execute Terminal commands from within Node.js. There is a promising ffmpeg WASM port that might make sense in the future. If you do not have ffmpeg installed, check out ffmpeg's download page.

exec(
  "ffmpeg -i Video/" +
    Name +
    " -ss 01:30 -r 1 -an -vframes 1 -f mjpeg Video/" +
    Name +
    ".jpg",
  function (err) {
    socket.emit("Done", { Image: "Video/" + Name + ".jpg" });
}

This ffmpeg command will generate one thumbnail at the 1:30 mark and save it to the Video/ folder with a .jpg file type. You can edit the time of the thumbnail by changing the -ss parameter. Once the thumbnail has been generated, we emit the Done event. Now, let's go back to the HTML page and implement it.

8. Finishing Up

The Done event will remove the progress bar and replace it with the thumbnail image. Because our server is not set up as a static file server, you have to place the location of your server (e.g. Apache) in the Path variable or configure the server to serve the images in order to load the image.

let Path = "https://localhost/";

socket.on("Done", function (data) {
    const Content = "Video Successfully Uploaded !!";
    Content +=
    "<img id='Thumb' src='" +
    Path +
    data["Image"] +
    "' alt='" +
    Name +
    "'><br>";
    Content +=
    "<button    type='button' name='Upload' value='' id='Restart' class='Button'>Upload Another</button>";
    document.getElementById("UploadArea").innerHTML = Content;
    document.getElementById("Restart").addEventListener("click", Refresh);
});

Above, we've added a button to begin uploading another file; all this does is refresh the page.

video uploader showing success and thumbnail

Conclusion

That's all there is to it, but, surely, you can imagine the possibilities when you pair this up with a database and an HTML5 player!

How to Create a Resumable Video Uploader in Node.js