AJAX Uploads

File uploads have long been an ugly spot in developing interfaces using HTML. Let's I've got a photo gallery. I don't know how many photos they're going to upload, so I don't know how many fields to show. Also, there is the possilibity of an error in the upload process, especially if the upload is sizable.

The solution I like the best is the attachment field in Google Mail's composition interface. Each time you add a new attachment, it creates a new file input. That's simple enough, but what is cool is that it uploads the file in the background without reloading the page. This gets around the all or nothing upload process. The problem is only that: I have no idea how to do this.

I guess the first place to start is with exactly which characteristics are avialable to the script that will be doing the uploading…

Props:

function printProperties(elementName, holderName) {
  element = document.getElementById(elementName);
  var count = 0;
  holder = document.getElementById(holderName);
  while(holder.childNodes.length > 0) holder.removeChild(holder.lastChild);
  for(prop in element) {
    if(count++ % 10 == 0) {
      table = document.createElement("table");
      holder.appendChild(table);
      header = document.createElement("tr");
      for(headerTitle in {"property":0, "type":1, "value":2}) {
        header.appendChild(document.createElement("th"));
        header.lastChild.appendChild(document.createTextNode(headerTitle));
      }
      table.appendChild(header)
    }
    row = document.createElement("tr");
    row.appendChild(document.createElement("td"));
    row.lastChild.appendChild(document.createTextNode(elementName + "[" + prop + "]"));
    row.appendChild(document.createElement("td"));
    try {
      row.lastChild.appendChild(document.createTextNode(typeof(element[prop])));
    } catch(exception) {
      row.lastChild.appendChild(document.createElement("em"));
      row.lastChild.lastChild.appendChild(document.createTextNode("exception"));
    }
    row.appendChild(document.createElement("td"));
    try {
      row.lastChild.appendChild(document.createTextNode(element[prop]));
    } catch(exception) {
      row.lastChild.appendChild(document.createTextNode(exception));
    }
    table.appendChild(row)
  }
}

Not much useful there that I can see… Joshua Eichorn uses the target property on the <form> to change an <iframe> and leave the main page unaffected. I want to try that:

The submission actually has to go somewhere, so I made a little php page to show the results of the submission:

Target <iframe>:

The problem is this page isn't called until after the file is completely uploaded. It is easy enough to deal with uploads, but so far as tracking progress, what can you do?

What I'd like to do is to start the upload and then post XMLHttpRequests to a SOAP service that could return the file's size. The issue is two fold. Though I can use the files' general format to guess which files in the upload temp directory are current uploads, there is no way to know which file is associated with which upload. The second problem is that even if I were to match a file with an upload, how do I know the total file size so as to calculate a completion percentage?

From looking on the internet, these problems are dealt with in three main fashions:

Simply take the newest file matching the filename format for an upload and assume that it is the upload. So far as knowing the total size, you don't.
Patch the php server to allow access to the post headers before the upload is complete.
Do the upload with a perl script which gives better access to the post parameters then hand off the rest of the handling to a php script.

I don't really like any of these:

This doesn't scale at all and doesn't provide for real progress.
Many people are not administrators on the systems that host their sites. They don't have the option of patching the server and if they do it keeps them from being able to update automatically.
This one is the best, but it requires perl which I have promised not to use after some non-web scripts were acidentally executed because they were in my web hierarchy.

Is it not possible to do this in straight php? I can't use php's upload processing because it doesn't let me know the temporary file names or the total file size. Could I use php to do the uploads on my own? PHP has socket functions. Could I use those to recieve the POST and then pass the information on to another script?

The problems are going to be twofold. One is how to execute the script to open the socket. Normally scripts are called in conjunction with a call to the server, but this script needs to be running simultaneously with the upload. Seems like I have two options:

Asynchronous: use php's exec function to kick off a server in the background. The server could handle multiple connections and do checking so there's only a single instance running. There is no particular reason that the server need be written in php other than it is likely installed on the server being used.
Synchronous: I could use a couple XMLHttpRequests to start the server and pass back a port number to connect to. This would keep everything running within Apache, but it seems a little complex since each upload would have to terminate and so different sockets would have to be used. What would the process look like?
1. XMLHttpRequest to get a port number (and the port would have to be reserved somehow)
2. XMLHttpRequest to start the server
3. HTTP POST of the uploaded information to the given port
4. multiple XMLHttpRequests to track the progress
5. the previously started server would end and the XMLHttpRequest would return
Ports can't be shared because the server has to end. Also, there are limits on execution time for scripts though and I'm not even sure that the upload server script would be allowed to complete.

All in all, it might be possible, but it doesn't seem to be the best solution in an environment that offers the ability to use the asynchronous option.

HTTP POSTS

In either case I have to handle the POSTs which I know pretty much nothing about. One simple way to start is to just point a form at the locl machine and see what comes though.

Quick question, does anyone know how to do away with the mkfifo call in this code?

mkfifo web_loop;
cat web_loop | nc -lp 8080 > web_loop;

Sample Post Trace:

POST /test_upload.php?id=identifier HTTP/1.1
Host: localhost:8080
User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.0.2) Gecko/20060308 Firefox/1.5.0.2
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: http://dysbulic.intranet/sites/himinbi.org/odin/ajax_uploader/
Content-Type: multipart/form-data; boundary=---------------------------1019292671580723810704877633
Content-Length: 544

-----------------------------1019292671580723810704877633
Content-Disposition: form-data; name="id"

identifier
-----------------------------1019292671580723810704877633
Content-Disposition: form-data; name="file"; filename="test.txt"
Content-Type: text/plain

Test text... Wisom some mime in it:
-----------------------------147483316912648177091998097157
Content-Disposition: form-data; name="id"

identifier
-----------------------------147483316912648177091998097157--

-----------------------------1019292671580723810704877633--

This looks pretty straightforward. HTTP headers are one per line with continuations allowed if they are whitespace indented. They are separated from the body by a blank line. The body then just has boundaries followed by some more headers and then the content. Simple enough. One possible issue is that the Content-Length is for the whole POST and not just the file bit. There could be multiple files in the same post. It is possible to include a Content-Length header on the file entry but Firefox at least doesn't seem to use it.

So long as only one file is uploaded at a time though the total size and file size should be relatively close.

Simple Upload Server:

While the upload is being processed, I want to be able to query as to it's status. I was thinking I'd use SOAP, but to do the query I need some way to identify the upload. The user should only be on one upload page at a time, so I can embed a UID in the form.

AJAX Uploads

Props:

Target <iframe>:

HTTP POSTS

Sample Post Trace:

Simple Upload Server:

Target `<iframe>:`