cURL is a package that contains various tools for transferring data between remote servers. It supports FTP, Windows Shares, Mail Servers, and of course Web Servers using HTTP.
Downloading a file from the Linux shell is usually accomplished using the cURL command like so:
curl http://example.org/file.zip --output file.zip
This makes the request for the file using the GET method and simply downloads it.
This article will detail how to use cURL to make a POST request, including form data. This may be useful if the server requires credentials to download a file, or if you are building an automated script to submit data to a web form for storage in a database.
It will also touch on submitting XML or JSON data from a file for use with a REST API.
POSTing URL Encoded Fields using cURL
For our examples, we will be submitting the following form data:
| Field Name | Value |
|---|---|
| field1 | value1 |
| field2 | value2 |
The data will be submitted to the URL https://example.org/submit.php. Addresses on example.org do not really exist, it’s just a web domain we can safely use for examples. submit.php is also an example name for a form submit script and doesn’t reference anything specific.
Form data can be passed to the POST request as a single string with each form field and its value defined by name=value, and separating them with & using the –data option:
curl --data "field1=value1&field2=value2" https://example.org/submit.php
However, this is not ideal, as it requires URL encoded data – that is, data which has had the spaces and slashes replaced with special codes to format them for sending over HTTP. You could do this manually, but cURL can do it for you using –data-urlencode:
curl --data-urlencode "field1=value1&field2=value2" https://example.org/submit.php
Field names are not URL encoded – if you wish to encode them prefix them with an =:
curl --data-urlencode "=user name=Linus Torvalds" http://example.org
We’ll use –data-urlencode for the following examples instead of –data for making POST requests to make things easy.
URL Encoded Fields Specified Separately
Each form field name/value can be passed separately – it makes things much simpler to read:
curl --data-urlencode "field1=value1" --data-urlencode "field2=value2" https://example.org/submit.php
Multipart/File Upload using cURL
Multipart form requests, including files, can be submitted via POST using the –form option. The below example will upload myfile.txt to the fileupload form field:
curl --form "[email protected]" https://example.org/submit.php
Multipart with Fields and Files, and Filenames
The form field name and value pairs can also be included with a file upload, and a filename for an uploaded file if you wish to change it:
curl --form "[email protected];filename=newfilename.txt" --form field1=value1 --form field2=value2 https://example.org/submit.php
Without Data
To make a POST request without any data, simply pass an empty string:
curl --data-urlencode '' https://example.org/submit.php
Showing File Upload Progress
To keep an eye on how an upload is progressing, add an -o output option to show a progress bar:
curl --tr-encoding -X POST -v -# -o output -T myfile.dat http://example.org/submit.php
Submitting Data Stored in a File
If uploading data already stored in a file, it can all be submitted at once, rather than parsing it out into a cURL command:
curl --data-urlencode [email protected] http://example.org/submit.php
JSON and XML data can be POSTed for programmatic API services by adding the appropriate headers:
curl -X POST --data-urlencode @myfile.txt http://example.org/submit.php --header "Content-Type:text/xml" curl -X POST --data-urlencode @myfile.txt http://example.org/submit.php --header "Content-Type:application/json"
Conclusion
Mastering cURL is important if you are frequently working on the Linux command line. It can perform many tasks for which you might usually need a full web browser, in addition to its uses interacting with programmatic APIs and for transferring data to and from your own servers.
Check out our other Linux shell and scripting articles here!
