The Purpose Driver

Curl: The Portable Internet Transfer Tool

Stephen Odogwu's photo
Stephen Odogwu
·

18 min read

Curl: The Portable Internet Transfer Tool

A lot of developers who have worked with REST APIs, have used tools like Postman and Hoppscotch, but only a few know about alternatives that can do the same job.

This article is one for getting started with the Curl command line tool, often written as cURL. A tool that has been for decades, which was born from an open source contribution to a pre-existing tool.

Prerequisites

  • Knowledge and understanding of the REST API.

  • Knowledge and understanding of HTTP.

  • Ability to use the Command-Line interface (CLI).

  • Previous usage of Postman or a similar tool.

Introduction

cURL stands for client URL. It is an open source project that makes a command line tool and a library libcurl for transferring data using internet protocols. It supports over 20 protocols, among which are HTTP and FTP.

cURL was first released in 1998 by Daniel Stenberg. It is an offshoot of a tool named httpget developed by Rafael Sagula.

In this article, I will be focusing specifically on the curl command line tool and its options. Also how we can use it for API calls and authentication by harnessing the HTTP methods.

Characteristics

There are several attributes that I touched on below, which curl possesses that has made it valuable in web development.

  • It is open source.

  • It is language agnostic : meaning that it supports a variety of programming languages.

  • Cross-Platform Compatible: It runs on a vast array of operating systems. Linux, Windows, Mac OS and more all support the use of curl without significant modifications.

  • It is backwards compatible: It has a characteristic that makes it compatible with older versions and with older hardware. It means we can use files created in previous versions in a new version without issues.

Installation

Below is a step by step guide to installing curl on various operating systems.

  • Windows method 1 If you are on Windows 10 version 1803 or later, it is already built into your OS. So you don't need to download anything. Default path is C:\Windows\System32\curl.exe.

  • Windows method 2

Steps to set-up

  1. Enter and access the URL https://curl.se/dlwiz/?type=bin to download the curl executable wizard.

  2. Under the "Select Operating System" section, select windows.

  3. Next select the parameters applicable to your version of windows.

  4. Once you've finished the on-screen steps, download the ZIP file generated. To download it, simply press "Download".

  5. Next, open the .zip file and enter to the folder called "src". Inside the src folder you will find the curl executable file. At this point, you need to copy the executable file and paste it inside a local folder on your PC to be able to run curl.

  6. From the Command Prompt, enter to the location where the executable file was pasted with the cd command. Now run the command below to verify the version of curl you have.

curl -v
  • Windows method 3 : Download git for Windows and it will install curl with it.

Steps to set-up

  1. You can find curl installed under c:\Program Files\Git\mingw64\bin\.

  2. Add it to your Windows path and you will be able to execute it from anywhere.Follow these steps to do so.

    a. In the search bar by the start menu, enter "Edit the system environment variables".

    b. Click on environment variables.

    c. Click on New.

    d. In the variable name field, enter curl.

    e. In the variable value field, enter c:\Program Files\Git\mingw64\bin\curl.exe.

    f. Click on Ok to exit.

    g. Enter curl --version in your command line to confirm you have it installed.

  • Linux method You do not need to download curl on Linux. It is built-in by default. However if it is not installed follow the steps below to set up curl on Linux.

Steps to set-up

  1. Open the terminal

  2. Run the command sudo apt-get install curl

  3. If a password is required after running the command, enter your computers' user password to continue. Then, wait until the installation finishes.

curl is now available for use on your pc.

  • Mac OS method: In the MacOs curl is built-in by default. So we don't need to download it. However to install it, we use the brew(Homebrew) package manager.

Steps to set-up

  1. Open your terminal.

  2. Run the command brew install curl

You now have curl installed on your macOS.

Command-Line Options

Curl, sometimes referred to as cURL offers a vast array of options, often called flags. These options, numbering over 200, are essential building blocks for constructing curl commands. When these commands are paired with HTTP methods, they become powerful tools for data transfer.

Given the extensive number of options, which exceeds 200, it is impractical to delve into each curl command individually. Instead, this discussion will focus on some of the most crucial options. Several of these options are essential for specific use cases, such as managing authentication and cookies.

These options usually have a dash or two before them. They have a single dash when an alphabet is used and two dashes when they are spelt out. Let us look at the options and their implementations to make commands below.

  • -V or --version: The curl -V( with an uppercase v) or the curl --version are used to show the cURL version which we have installed. See the command and response below.
C:\Users\user>curl -V
curl 8.0.1 (Windows) libcurl/8.0.1 Schannel WinIDN
Release-Date: 2023-03-20
Protocols: dict file ftp ftps http https imap imaps pop3 pop3s smtp smtps telnet tftp
Features: AsynchDNS HSTS HTTPS-proxy IDN IPv6 Kerberos Largefile NTLM SPNEGO SSL SSPI threadsafe Unicode UnixSockets
  • -X: curl by default sends a GET request. We use -X to make our request explicit with the format curl -X <method> URL. Below I am going to illustrate two GET requests with the curl command. One with the -X and the other without the -X, both fetching data from our URL @ https://my-burger-api.herokuapp.com/burgers.
C:\Users\user>curl -X GET https://my-burger-api.herokuapp.com/burgers
[
  {
    "id": 0,
    "name": "Tribute Burger",
    "restaurant": "Honest Burgers",
    "web": "www.honestburgers.co.uk",
    "description": "A mouth-watering honest beef burger",
    "ingredients": [
      "beef",
      "american cheese",
      "burger sauce",
      "french mustard",
      "pickes",
      "onion",
      "lettuce"
    ],
    "addresses": [
      {
        "addressId": 0,
        "number": "75",
        "line1": "Venn Street",
        "line2": "Clapham",
        "postcode": "SW4 0BD",
        "country": "United Kingdom"
      }
    ]
  },
  {
    "id": 1,
    "name": "Pulled Mooshie",
    "restaurant": "Mooshies",
    "web": "www.veganburger.org",
    "description": "Spicy vegan burger with jackfruit",
    "ingredients": [
      "jackfruit",
      "coleslaw",
      "gluten free bun"
    ],
    "addresses": [
      {
        "addressId": 0,
        "number": "104",
        "line1": "Brick Lane",
        "line2": "Shoreditch",
        "postcode": "E1 6RL",
        "country": "United Kingdom"
      }
    ]
  }
  ]

C:\Users\user>curl https://my-burger-api.herokuapp.com/burgers
[
  {
    "id": 0,
    "name": "Tribute Burger",
    "restaurant": "Honest Burgers",
    "web": "www.honestburgers.co.uk",
    "description": "A mouth-watering honest beef burger",
    "ingredients": [
      "beef",
      "american cheese",
      "burger sauce",
      "french mustard",
      "pickes",
      "onion",
      "lettuce"
    ],
    "addresses": [
      {
        "addressId": 0,
        "number": "75",
        "line1": "Venn Street",
        "line2": "Clapham",
        "postcode": "SW4 0BD",
        "country": "United Kingdom"
      }
    ]
  },
  {
    "id": 1,
    "name": "Pulled Mooshie",
    "restaurant": "Mooshies",
    "web": "www.veganburger.org",
    "description": "Spicy vegan burger with jackfruit",
    "ingredients": [
      "jackfruit",
      "coleslaw",
      "gluten free bun"
    ],
    "addresses": [
      {
        "addressId": 0,
        "number": "104",
        "line1": "Brick Lane",
        "line2": "Shoreditch",
        "postcode": "E1 6RL",
        "country": "United Kingdom"
      }
    ]
  }
  ]

Note: The responses which I have received and will be receiving from our sample burger URL are more than I display in the code. I had to minimize space, so I didn’t display all. You can try the same request to see the full responses.

  • -i or --include: This option is one we add when we want our HTTP response headers included as part of our output. The headers are displayed above the body. Below is how we use it to carry out a command for a GET request to our sample URL and our response.
C:\Users\user>curl -i https://my-burger-api.herokuapp.com/burgers
HTTP/1.1 200 OK
Server: Cowboy
Report-To: {"group":"heroku-nel","max_age":3600,"endpoints":[{"url":"https://nel.heroku.com/reports?ts=1698313173&sid=e11707d5-02a7-43ef-b45e-2cf4d2036f7d&s=sPATZ5KYepOdWKwPamcdwPn4GJk5l%2BR9WBOmqw52K9g%3D"}]}
Reporting-Endpoints: heroku-nel=https://nel.heroku.com/reports?ts=1698313173&sid=e11707d5-02a7-43ef-b45e-2cf4d2036f7d&s=sPATZ5KYepOdWKwPamcdwPn4GJk5l%2BR9WBOmqw52K9g%3D
Nel: {"report_to":"heroku-nel","max_age":3600,"success_fraction":0.005,"failure_fraction":0.05,"response_headers":["Via"]}
Connection: keep-alive
X-Powered-By: Express
Vary: Origin, Accept-Encoding
Access-Control-Allow-Credentials: true
Cache-Control: no-cache
Pragma: no-cache
Expires: -1
X-Content-Type-Options: nosniff
Content-Type: application/json; charset=utf-8
Content-Length: 20826
Etag: W/"515a-nWi+LSEz+N/8zkJUxW8tpZwF4y0"
Date: Thu, 26 Oct 2023 09:39:33 GMT
Via: 1.1 vegur

[
  {
    "id": 0,
    "name": "Tribute Burger",
    "restaurant": "Honest Burgers",
    "web": "www.honestburgers.co.uk",
    "description": "A mouth-watering honest beef burger",
    "ingredients": [
      "beef",
      "american cheese",
      "burger sauce",
      "french mustard",
      "pickes",
      "onion",
      "lettuce"
    ],
    "addresses": [
      {
        "addressId": 0,
        "number": "75",
        "line1": "Venn Street",
        "line2": "Clapham",
        "postcode": "SW4 0BD",
        "country": "United Kingdom"
      }
    ]
  },
  {
    "id": 1,
    "name": "Pulled Mooshie",
    "restaurant": "Mooshies",
    "web": "www.veganburger.org",
    "description": "Spicy vegan burger with jackfruit",
    "ingredients": [
      "jackfruit",
      "coleslaw",
      "gluten free bun"
    ],
    "addresses": [
      {
        "addressId": 0,
        "number": "104",
        "line1": "Brick Lane",
        "line2": "Shoreditch",
        "postcode": "E1 6RL",
        "country": "United Kingdom"
      }
    ]
  }
]
  • -I: We use this option as part of our command when we want only the response headers returned back. It is particularly useful when we want to inspect the headers to determine if we will download the body content with a GET request.
C:\Users\user>curl -I https://my-burger-api.herokuapp.com/burgers
HTTP/1.1 200 OK
Server: Cowboy
Report-To: {"group":"heroku-nel","max_age":3600,"endpoints":[{"url":"https://nel.heroku.com/reports?ts=1698313173&sid=e11707d5-02a7-43ef-b45e-2cf4d2036f7d&s=sPATZ5KYepOdWKwPamcdwPn4GJk5l%2BR9WBOmqw52K9g%3D"}]}
Reporting-Endpoints: heroku-nel=https://nel.heroku.com/reports?ts=1698313173&sid=e11707d5-02a7-43ef-b45e-2cf4d2036f7d&s=sPATZ5KYepOdWKwPamcdwPn4GJk5l%2BR9WBOmqw52K9g%3D
Nel: {"report_to":"heroku-nel","max_age":3600,"success_fraction":0.005,"failure_fraction":0.05,"response_headers":["Via"]}
Connection: keep-alive
X-Powered-By: Express
Vary: Origin, Accept-Encoding
Access-Control-Allow-Credentials: true
Cache-Control: no-cache
Pragma: no-cache
Expires: -1
X-Content-Type-Options: nosniff
Content-Type: application/json; charset=utf-8
Content-Length: 20826
Etag: W/"515a-nWi+LSEz+N/8zkJUxW8tpZwF4y0"
Date: Thu, 26 Oct 2023 09:39:33 GMT
Via: 1.1 vegur
  • --json: We use this to send specified json data to a URL. Requests like the POST and PUT usually employ this option. We don't need to specify the method or send Content-Type:application/json for header as it is implied by the --json option. The command format is curl --json <data> URL. We can bundle our data in two ways to send.

By writing the data explicitly.

curl --json '{"key1":"value1","key2":"value2"}' https://my-burger-api.herokuapp.com/burgers

Secondly we can place our data in a file with a json extension as burger.json. We can send it like this below.

curl --json @/full/path/to/burger.json https://my-burger-api.herokuapp.com/burgers
  • -H or --header: We use this to submit our request headers to the URL. When sending headers like Content-Type,Accept and Authorization, we use this.
curl -H "Accept: application/json" https://my-burger-api.herokuapp.com/burgers
  • -d or --data: Similar to --json which we discussed above. We use this to send data to our URL. This works with the POST,PATCH and PUT HTTP methods. When using this we add the -H option with it in order to specify the Content-Type as a header.
curl -d '{"key1": "value1","key2":"value2" }' -H "Content-Type: application/json" https://my-burger-api.herokuapp.com/burgers

When using Windows command prompt, we use double quotes to encapsulate the argument. Inner double quotes are also escaped. See example below.

curl -d "{\"key\":\"value\", \"key\":\"value\"}" -H "Content-Type: application/json" https://my-burger-api.herokuapp.com/burgers

We can also place the data in a file with a json extension to send it.

curl -d @/full/path/to/burger.json -H "Content-Type: application/json" https://my-burger-api.herokuapp.com/burgers

However when we want to send a PUT or PATCH request, we have to be explicit.

curl -d '{"key1": "value1","key2":"value2" }' -H "Content-Type: application/json" -X PUT https://my-burger-api.herokuapp.com/burgers/7
  • -v or --verbose: A lowercase v in this scenario. We use this option to include the request headers and response headers as part of our response alongside the body. It shows detailed information about our request and response. See below.
 Host: my-burger-api.herokuapp.com
> User-Agent: curl/8.0.1
> Accept: */*
>
* schannel: failed to decrypt data, need more data
* schannel: failed to decrypt data, need more data
< HTTP/1.1 200 OK
< Server: Cowboy
< Report-To: {"group":"heroku-nel","max_age":3600,"endpoints":[{"url":"https://nel.heroku.com/reports?ts=1698313911&sid=e11707d5-02a7-43ef-b45e-2cf4d2036f7d&s=CVmqnZcTiDimL%2BcyjZIirsp6AG2dVa0cqvFtRgpwr00%3D"}]}
< Reporting-Endpoints: heroku-nel=https://nel.heroku.com/reports?ts=1698313911&sid=e11707d5-02a7-43ef-b45e-2cf4d2036f7d&s=CVmqnZcTiDimL%2BcyjZIirsp6AG2dVa0cqvFtRgpwr00%3D
< Nel: {"report_to":"heroku-nel","max_age":3600,"success_fraction":0.005,"failure_fraction":0.05,"response_headers":["Via"]}
< Connection: keep-alive
< X-Powered-By: Express
< Vary: Origin, Accept-Encoding
< Access-Control-Allow-Credentials: true
< Cache-Control: no-cache
< Pragma: no-cache
< Expires: -1
< X-Content-Type-Options: nosniff
< Content-Type: application/json; charset=utf-8
< Content-Length: 20826
< Etag: W/"515a-nWi+LSEz+N/8zkJUxW8tpZwF4y0"
< Date: Thu, 26 Oct 2023 09:51:51 GMT
< Via: 1.1 vegur
<
[
  {
    "id": 0,
    "name": "Tribute Burger",
    "restaurant": "Honest Burgers",
    "web": "www.honestburgers.co.uk",
    "description": "A mouth-watering honest beef burger",
    "ingredients": [
      "beef",
      "american cheese",
      "burger sauce",
      "french mustard",
      "pickes",
      "onion",
      "lettuce"
    ],
    "addresses": [
      {
        "addressId": 0,
        "number": "75",
        "line1": "Venn Street",
        "line2": "Clapham",
        "postcode": "SW4 0BD",
        "country": "United Kingdom"
      }
    ]
  }
]
  • -o: The curl -o URL command is used to create a file which downloaded content should be saved. These files could be varieties of extensions. see below.
curl -o myAudio.mp3 URL
curl -o myFile.txt URL

With our sample URL we can also download the body from a GET request. See below.

C:\Users\user>curl -o burgerjoints.json https://my-burger-api.herokuapp.com/burgers
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 20509  100 20509    0     0  12149      0  0:00:01  0:00:01 --:--:-- 12164

This downloads the body of the response to a file named burgerjoints.json on my pc.

  • -O: The capital o flag. This option helps us preserve the file name of the downloaded content.
curl -O URL/filename.mp4

The downloaded file will be saved as filename.

  • -F: This option is used to submit forms that contain both text data and files. With this we use curl to interact with web applications that accept form based data. Forms that involve uploading files. Such as videos, images, documents like multipart forms. See below.
curl -F "image:/path/to/image.jpg" -F "description: my pic"
  • -L or --location: The -L option with curl is an instruction to curl to follow any redirects automatically. This is for handling situations where web pages have been moved. It also comes in handy to access resources that use multiple URLs to load content.
curl -L https://my-burger-api.herokuapp.com/burgers

So if a server responds with a redirect (e.g. 302 status code) and provides a new URL, curl will make a request to the new URL and get the content from there. I have two scenarios for you to try out. Send two GET requests to https://api.currencyscoop.com/v1. One with a -L option and one without. Like these.

curl -L https://api.currencyscoop.com/v1

curl https://api.currencyscopp.com/v1

The above URL is one that has been changed to "https://api.currencybeacon.com".

While at it, I also want you to try any of them with the verbose option(-v) like this below. To get a verbose response.

C:\Users\user>curl -L https://api.currencyscoop.com/v1 -v
*   Trying 172.67.155.40:443...
* Connected to api.currencyscoop.com (172.67.155.40) port 443 (#0)
* schannel: disabled automatic use of client certificate
* ALPN: offers http/1.1
* ALPN: server accepted http/1.1
* using HTTP/1.1
> GET /v1 HTTP/1.1
> Host: api.currencyscoop.com
> User-Agent: curl/8.0.1
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently ⬅️
> 
< Date: Sat, 28 Oct 2023 02:43:00 GMT
< Transfer-Encoding: chunked
< Connection: keep-alive
< Cache-Control: max-age=3600
< Expires: Sat, 28 Oct 2023 03:43:00 GMT
< Location: https://api.currencybeacon.com/v1 ⬅️

< Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=AM3Oo5DX%2BSZzT9gdsb5c6vcOvpem%2FwREAKlxsfxyhgUB2Llk1N2mNWs%2FcsujfB3bs93lmj0sxleWh2ESnTneNtNdgkhvC6N4bHc50KHPk%2FJuZg35hKL3ZGjIN7Xe5vXTy%2FRuuiPAIX8%3D"}],"group":"cf-nel","max_age":604800}
< NEL: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
< Server: cloudflare
< CF-RAY: 81cfd42a3bb30ea7-AMS
< alt-svc: h3=":443"; ma=86400
<
* Ignoring the response-body
* Connection #0 to host api.currencyscoop.com left intact
* Issue another request to this URL: 'https://api.currencybeacon.com/v1' ⬅️

*   Trying 104.21.33.73:443...
* Connected to api.currencybeacon.com (104.21.33.73) port 443 (#1)
* schannel: disabled automatic use of client certificate
* ALPN: offers http/1.1
* ALPN: server accepted http/1.1
* using HTTP/1.1
> GET /v1 HTTP/1.1
> Host: api.currencybeacon.com
> User-Agent: curl/8.0.1
> Accept: */*

See information in places I marked with arrows above, from the --verbose response.

Authentication

While we have previously looked at some options for creating curl commands, there are a few that are multifaceted(even though they are mostly specialized).

In this section, I will be bringing back an option or two you have previously seen along with a new option to show how to perform authentication and authorization using curl commands.

  • -H or --header: Like we saw previously, this option is used to send headers while making requests. Among the headers which we didn't use for demonstration is the Authorization header. This header comes in handy when certain credentials are required to access a URL. Some of these could be web tokens, API keys e.t.c.

For web tokens, we sign in with a POST request like this.

curl --json '{"email":"youremail@mail.com","password":"your_password"} https://sampleURL.com

In the above, it might not be email though, might be username. When you sign in, you receive a token. This token will be used for Authorization like below.

curl - H "Authorization: Bearer your_token" https://sampleURL.com

For sites that require API keys, they usually have their own custom headers for this. These headers usually start with X- ( not to be confused with -X). See below.

curl -H "X-SiteAPI-key:your_key_here" https://sampleURL.com
  • -u or --user: The curl -u command takes a username and password when sending HTTP requests. This command is used for basic authentication when making HTTP requests. Below is how we use the command.
curl -u username:password https://sampleURL.com/protected/rrsource

When you specify the -u option, curl will include the Authorization header in the request to authenticate you with the server.

Please note that this form of authentication is not the most secure and it is typically used over HTTPS to help protect the credentials from being transmitted in plain text. More secure authentication methods like tokens are preferred, because the credentials are visible in the request when we don't use HTTPS.

Handling Cookies

Cookies are used for tracking user session data on the web. We can use curl to send and receive these cookies as curl provides options for managing these cookies. Let us now take a look at these options and how they can be implemented with curl commands.

  • -c or --cookie-jar: This option is used for saving cookies on the client or to set cookies as is the genarlly used term. These cookies are gotten from the server when the client makes a request to a server that has these cookies. To illustrate We typically send a file with a .txt extension along with the request. This file is where the received cookies will be saved. It takes the format below.
curl -c cookies.txt https://sampleURL.com
  • -b or --cookie: We use this option when we want to send the saved cookie in the client to the server. This is usually used for subsequent requests when the cookie must have been saved on the client. It takes the format below.
curl -b cookies.txt https://sampleURL.com

The cookies are read from the cookies.txt file to send.

we can also send specific cookies among the save cookies with this option. See below.

curl -b "cookieName=valueOfCookie" https://sampleURL.com

Below I want to make a simple app we can use to create this in real time.See steps below.

  1. Go to your vs code. Create a directory name it cookie-setter.

  2. create an app.js file within it.

  3. open up notepad and create a cookies.txt file

  4. open a terminal within vs code.

  5. Run npm init to create a package.json file.

  6. In your terminal, run npm install express.

  7. Next run npm install cookie-parser.

  8. After installing both packages go down to your app.js file and add the code below.

const express = require('express')
const app = express()
const cookieParser = require('cookie-parser')
app.use(cookieParser())
app.get('/setcookie',(req,res)=>{
    let maxAge = 24 * 60 *60* 1000 //milliseconds
 res.cookie('user','your_name',maxAge) 
  res.send('cookie has been set')
})
app.get('/getcookie',(req,res)=>{
    const myCookie = req.cookies.user
    res.json(`value of 'user' cookie:${myCookie}`)
})

let port = 4500
app.listen(4500,()=>{
    console.log(`server is running on port ${port}`)
})

Above we have a setcookie route which when we visit triggers the setting of cookies in our device. We also have a getcookie route which when we visit shows us the cookie we set.

Note:your package.json file should look like below.


  "name": "cookie-setter",
  "version": "1.0.0",
  "description": "",
  "main": "app.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "cookie-parser": "^1.4.6",
    "express": "^4.18.2"
  }

}

open up another terminal in vs code or open up your Command prompt. cd to the folder containing the cookies.txt file. Mine is in the Documents folder. Run the command below with the --verbose or -v flag.

C:\Users\user\Documents>curl -c Cookies.txt http://localhost:4500/setcookie -v

C:\Users\user\Documents>curl -c Cookies.txt http://localhost:4500/setcookie -v
*   Trying 127.0.0.1:4500...
* Connected to localhost (127.0.0.1) port 4500 (#0)
> GET /setcookie HTTP/1.1
> Host: localhost:4500
> User-Agent: curl/8.0.1
> Accept: */*
>
< HTTP/1.1 200 OK
< X-Powered-By: Express
* Added cookie user="steve" for domain localhost, path /, expire 0⬅️
< Set-Cookie: user=steve; Path=/
< Content-Type: text/html; charset=utf-8
< Content-Length: 19
< ETag: W/"13-UbqCgG6k5thPk0DHAkVScaoIJp8"
< Date: Sun, 29 Oct 2023 09:17:17 GMT
< Connection: keep-alive
< Keep-Alive: timeout=5
<
cookie has been set* Connection #0 to host localhost left intact

Above I visited the setcookie route which triggered the setting of cookies in our cookies.txt file. You can see the information at the line where the arrow is pointing to above.

Now let us visit the getcookie route to see our cookie on the server.Run command below.

C:\Users\user\Documents>curl -b Cookies.txt http://localhost:4500/getcookie -v

C:\Users\user\Documents>curl -b Cookies.txt http://localhost:4500/getcookie -v
*   Trying 127.0.0.1:4500...
* Connected to localhost (127.0.0.1) port 4500 (#0)
> GET /getcookie HTTP/1.1
> Host: localhost:4500
> User-Agent: curl/8.0.1
> Accept: */*
> Cookie: user=steve ⬅️
>
< HTTP/1.1 200 OK
< X-Powered-By: Express
< Content-Type: text/html; charset=utf-8
< Content-Length: 28
< ETag: W/"1c-8P5w3huyD6e9oztUcohhLAM8ogo"
< Date: Sun, 29 Oct 2023 09:25:39 GMT
< Connection: keep-alive
< Keep-Alive: timeout=5
<
value of 'user' cookie:steve* Connection #0 to host localhost left intact

From the line where I put an arrow, you can see my cookie header, as well as the key and value.

  • -H or --header: We can also use this option to create a cookie header which to send specific cookies.
curl -H "cookie: user=your_name"

URL Globbing

The practice of using using wildcard characters in URLs to match multiple resources at once, and get the content of those resources. URL globbing allows us specify sets of URLs using special characters like square brackets [] and curly braces {}. These characters are usually not allowed in standard URLs, except in specific cases like Internet Protocol Version 6 (IPV6) addresses.

curl leverages these characters for its own URL globbing feature, allowing users to specify ranges and sets of URLs more conveniently. We will now take a look at specific ways to use these characters.

  • Square Brackets []: It is used to specify a range of characters. See an illustration below of a way we can use it.
curl http://localhost:4500/folder/file[1-3].html

From the above, curl will retrieve files like file1.html, file2.html, and file3.html. If there is a file4.html, it won't retrieve it.

  • Curly Braces {}: It is used to specify a set of characters.The characters are usually separated by commas. See an illustration below of a way it can be used.
curl http://localhost:4500/folder/file{1,2,5}.html

From the above curl will retrieve file1.html, file2.html and file5.html.

Download Content We might need to download the content from the URLs we accessed. We add a -O flag. This should download content from the specified URLS. See the code below.

curl -O http://localhost:4500/folders/{index,support,about}.html

This downloads content from index.html, support.html and about.html.

Literal Characters But we know that in IPV6 addresses we might find curly braces and square brackets as part of url characters. We will now take a look at ways to handle such.

  • Turning off globbing: If the square brackets or curly braces appear in a URL and aren't intended for globbing, you can disable globbing by using the --globoff or -g flag with the curl command.

  • URL Encoding: Globbing characters must be URL encoded if we want to use them as literal characters in the URL. If we want ti use square brackets or curly braces as real URL characters, we need to encode them using URL encoding.

In this URL encoding, we replace special characters with a percent sign "%" followed by two hexadecimal digits representing the ASCII code of the character. See code lists below.

  • [ becomes %5B

  • ] becomes %5D

  • { becomes %7B

  • } becomes %7D

Note: Ensure your Curl version supports URL globbing, as older versions may not have this feature.

Conclusion

We have learnt abouthow to execute HTTP requests with curl. The following links were instrumental to my understanding of the tool. You can study them to gain more clarity.

REST APIhttphttpsServer side rendering