Uploading large HTTP multipart request with System.Net.HttpWebRequest in C#

My previous post described a method of sending a file and some data via HTTP multipart post by constructing the HTTP request with the System.IO.MemoryStream class before writing the contents to the System.Net.HttpWebRequest class. This method of sending our HTTP request will work only if we can restrict the total size of our file and data.

Although the MemoryStream class reduces programming effort, using it to hold a large amount of data will result in a System.OutOfMemoryException being thrown. Hence, to send large amount of data, we will need to write our contents to the HttpWebRequest instance directly. Before doing so, there are several properties in the HttpWebRequest instance that we will need to set.

HttpWebRequest requestToServer = (HttpWebRequest)
    WebRequest.Create("http://localhost/fileReceiver.php");

// Define a boundary string
string boundaryString = "----SomeRandomText";

// Turn off the buffering of data to be written, to prevent
// OutOfMemoryException when sending data
requestToServer.AllowWriteStreamBuffering = false;
// Specify that request is a HTTP post
requestToServer.Method = WebRequestMethods.Http.Post;
// Specify that the content type is a multipart request
requestToServer.ContentType 
    = "multipart/form-data; boundary=" + boundaryString;
// Turn off keep alive
requestToServer.KeepAlive = false;

Calculating the total size of the upload content

Recall that a HTTP multipart post request resembles the following form:

POST http://127.0.0.1/GetPostRequest.php HTTP/1.1
Host: 127.0.0.1
Connection: keep-alive
Referer: http://localhost/GetPostRequest.php
Content-Length: 1611568
Cache-Control: max-age=0
Origin: http://localhost
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/534.30 (KHTML, like Gecko) Chrome/12.0.742.91 Safari/534.30
Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryX6nBO7q27yQ1JNbb
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3

------WebKitFormBoundaryX6nBO7q27yQ1JNbb
Content-Disposition: form-data; name="myFileDescription"

 My sample file description.
------WebKitFormBoundaryX6nBO7q27yQ1JNbb
Content-Disposition: form-data; name="myFile"; filename="SomeRandomFile.pdf"
Content-Type: application/pdf

file contents...
------WebKitFormBoundaryX6nBO7q27yQ1JNbb--

From the HTTP request created by the browser, we see that the upload content spans from the first boundary string  to the last boundary string. The total size of this block of content need to be set to the ContentLength property of the HttpWebRequest instance, before we write any data out to the request stream. To calculate the total size of the HTTP request, we need to add the byte sizes of the string values and the file that we are going to upload. We can convert the strings in the HTTP request into byte arrays with the System.Text.ASCIIEncoding class and get the size of the strings with the Length property of the byte arrays. The size of the file can be retrieved via the Length property of a System.IO.FileInfo instance.

ASCIIEncoding ascii = new ASCIIEncoding();
string boundaryStringLine = "\r\n--" + boundaryString + "\r\n";
byte[] boundaryStringLineBytes = ascii.GetBytes(boundaryStringLine);

string lastBoundaryStringLine = "\r\n--" + boundaryString + "--\r\n";
byte[] lastBoundaryStringLineBytes = ascii.GetBytes(lastBoundaryStringLine);

// Get the byte array of the myFileDescription content disposition
string myFileDescriptionContentDisposition = String.Format(
    "Content-Disposition: form-data; name=\"{0}\"\r\n\r\n{1}",
    "myFileDescription",
    "A sample file description");
byte[] myFileDescriptionContentDispositionBytes 
    = ascii.GetBytes(myFileDescriptionContentDisposition);

string fileUrl = @"C:\SomeRandomFile.pdf";
// Get the byte array of the string part of the myFile content
// disposition
string myFileContentDisposition = String.Format(
    "Content-Disposition: form-data;name=\"{0}\"; "
     + "filename=\"{1}\"\r\nContent-Type: {2}\r\n\r\n",
    "myFile", Path.GetFileName(fileUrl), Path.GetExtension(fileUrl));
byte[] myFileContentDispositionBytes =
    ascii.GetBytes(myFileContentDisposition);

FileInfo fileInfo = new FileInfo(fileUrl);

// Calculate the total size of the HTTP request
long totalRequestBodySize = boundaryStringLineBytes.Length * 2
    + lastBoundaryStringLineBytes.Length
    + myFileDescriptionContentDispositionBytes.Length
    + myFileContentDispositionBytes.Length
    + fileInfo.Length;
// And indicate the value as the HTTP request content length
requestToServer.ContentLength = totalRequestBodySize;

Sending the HTTP request content

After calculating the content length, we can write the byte arrays that we have generated previously to the stream returned via the HttpWebRequest.GetRequestStream() method.

// Write the http request body directly to the server
using (Stream s = requestToServer.GetRequestStream())
{
    // Send the file description content disposition over to the server
    s.Write(boundaryStringLineBytes, 0, boundaryStringLineBytes.Length);
    s.Write(myFileDescriptionContentDisposition , 0,
        myFileDescriptionContentDisposition.Length);

    // Send the file content disposition over to the server
    s.Write(boundaryStringLineBytes, 0, boundaryStringLineBytes.Length);
    s.Write(myFileContentDispositionBytes, 0,
        myFileContentDispositionBytes.Length);

    // Send the file binaries over to the server, in 1024 bytes chunk
    FileStream fileStream = new FileStream(fileUrl, FileMode.Open,
        FileAccess.Read);
    byte[] buffer = new byte[1024];
    int bytesRead = 0;
    while ((bytesRead = fileStream.Read(buffer, 0, buffer.Length)) != 0)
    {
        s.Write(buffer, 0, bytesRead);
    } // end while
    fileStream.Close();

    // Send the last part of the HTTP request body
    s.Write(lastBoundaryStringLineBytes, 0, lastBoundaryStringLineBytes.Length);
} // end using

Getting the response from the server

We get the server response by reading from the System.Net.WebResponse instance, that can be retrieved via the HttpWebRequest.GetResponseStream() method.

// Grab the response from the server. WebException will be thrown
// when a HTTP OK status is not returned
WebResponse response = requestToServer.GetResponse();
StreamReader responseReader = new StreamReader(response.GetResponseStream());
string replyFromServer = responseReader.ReadToEnd();

Related posts

• Downloading a file from a HTTP server with System.Net.HttpWebRequest in C#
• Handling web server communication feedback with System.Net.WebException in C#
• Sending a file and some form data via HTTP post in C#
• How to build a web based user interaction layer in C#

• Share
• Tweet
• Share
• Save