How to Import JSON from a Bash Script on Linux

Avatar

By squashlabs, Last Updated: October 21, 2023

How to Import JSON from a Bash Script on Linux

JSON (JavaScript Object Notation) is a lightweight data interchange format widely used for representing structured data. It is often used for configuration files, API responses, and data exchange between systems. In this article, we will explore how to import data from a JSON file into a Bash script on Linux.

There are several ways to accomplish this task, depending on the specific requirements of your script. We will cover different techniques, libraries, and tools that can be used to parse and import JSON data into a Bash script.

Parsing JSON in Bash

To parse JSON in Bash, we need a tool or library that can understand the JSON format and provide functions or utilities to extract data from it. One popular tool for parsing JSON in Bash is jq.

jq is a lightweight and flexible command-line JSON processor. It allows you to filter, transform, and manipulate JSON data using a simple and expressive syntax. With jq, you can easily extract specific values, filter data based on conditions, and perform complex operations on JSON objects.

Example 1: Parsing JSON with jq

Let’s say we have a JSON file named data.json with the following content:

{
  "name": "John Doe",
  "age": 30,
  "email": "john.doe@example.com",
  "address": {
    "street": "123 Main St",
    "city": "New York",
    "state": "NY"
  }
}

We can use jq to extract the value of the name field from this JSON file:

name=$(jq -r '.name' data.json)
echo $name

Output:

John Doe

In this example, we use the -r option to output the result as raw text, without quotes. The . in the jq command represents the root object of the JSON file, and .name is the path to the name field.

Example 2: Filtering JSON with jq

jq also allows us to filter JSON data based on conditions. Let’s consider a JSON file named users.json with an array of user objects:

[
  {
    "name": "John Doe",
    "age": 30,
    "email": "john.doe@example.com"
  },
  {
    "name": "Jane Smith",
    "age": 25,
    "email": "jane.smith@example.com"
  },
  {
    "name": "Bob Johnson",
    "age": 35,
    "email": "bob.johnson@example.com"
  }
]

We can use jq to filter this JSON file and select only the users with an age greater than 30:

jq '.[] | select(.age > 30)' users.json

Output:

{
  "name": "John Doe",
  "age": 30,
  "email": "john.doe@example.com"
}
{
  "name": "Bob Johnson",
  "age": 35,
  "email": "bob.johnson@example.com"
}

In this example, we use the .[] syntax to iterate over each element in the array, and select(.age > 30) to filter the objects based on the condition.

Related Article: How To Echo a Newline In Bash

Reading JSON in Bash

Now that we know how to parse JSON using jq, let’s explore different techniques to read JSON data in Bash. Reading JSON can be done in two ways: reading from a file or reading from a JSON string.

Reading from a File

To read JSON data from a file, we can use the cat command along with jq to parse and process the JSON content. Here’s an example:

json=$(cat data.json)
name=$(echo $json | jq -r '.name')
echo $name

Output:

John Doe

In this example, we use the cat command to read the content of the data.json file and store it in the json variable. Then, we use echo to pass the JSON data to jq for parsing and extract the name field.

Reading from a JSON String

If you have the JSON data as a string, you can directly pass it to jq without the need to read from a file. Here’s an example:

json='{
  "name": "John Doe",
  "age": 30,
  "email": "john.doe@example.com"
}'

name=$(echo $json | jq -r '.name')
echo $name

Output:

John Doe

In this example, we assign the JSON data to the json variable as a string. Then, we use echo to pass the JSON data to jq for parsing and extract the name field.

Converting JSON to Bash Variables

Once we have parsed the JSON data, we can convert it into Bash variables for further processing within our script. There are different approaches to achieve this, depending on the structure of the JSON data.

Converting JSON Objects to Bash Variables

If the JSON data represents an object, we can convert it into Bash variables using the declare command. Here’s an example:

json='{
  "name": "John Doe",
  "age": 30,
  "email": "john.doe@example.com"
}'

declare $(echo $json | jq -r 'to_entries | .[] | "export \(.key)=\(.value)"')
echo $name
echo $age
echo $email

Output:

John Doe
30
john.doe@example.com

In this example, we use the to_entries filter in jq to convert the JSON object into an array of key-value pairs. Then, we iterate over each pair using .[] and construct a Bash variable assignment expression using "export \(.key)=\(.value)". Finally, we use the declare command to evaluate and assign the Bash variables.

Converting JSON Arrays to Bash Variables

If the JSON data represents an array, we can convert it into Bash variables using a loop. Here’s an example:

json='[1, 2, 3, 4, 5]'

i=0
for value in $(echo $json | jq -r '.[]'); do
  declare -i "var$i=$value"
  ((i++))
done

echo $var0
echo $var1
echo $var2
echo $var3
echo $var4

Output:

1
2
3
4
5

In this example, we use the .[] syntax in jq to iterate over each element in the array. Then, we use a loop in Bash to assign each value to a Bash variable using the declare command.

Techniques for Importing JSON Data into a Linux Bash Script

There are several techniques you can use to import JSON data into a Linux Bash script, depending on your specific requirements and constraints. In this section, we will explore some of the common techniques used in practice.

Related Article: How to Use If-Else Statements in Shell Scripts

Technique 1: Reading JSON from a File

One of the simplest and most common techniques is to read JSON data from a file using the cat command and parse it using jq. Here’s an example:

json=$(cat data.json)
name=$(echo $json | jq -r '.name')
age=$(echo $json | jq -r '.age')
email=$(echo $json | jq -r '.email')

echo $name
echo $age
echo $email

Output:

John Doe
30
john.doe@example.com

In this example, we use the cat command to read the content of the data.json file and store it in the json variable. Then, we use echo to pass the JSON data to jq for parsing and extract the desired fields.

Technique 2: Using Command Substitution

Another technique is to use command substitution to directly assign the result of a command to a variable. Here’s an example:

name=$(jq -r '.name' data.json)
age=$(jq -r '.age' data.json)
email=$(jq -r '.email' data.json)

echo $name
echo $age
echo $email

Output:

John Doe
30
john.doe@example.com

In this example, we use the jq command to directly extract the desired fields from the data.json file and assign them to variables using command substitution.

Technique 3: Passing JSON as a Command-Line Argument

If you have a small JSON payload, you can pass it as a command-line argument to your Bash script. Here’s an example:

#!/bin/bash

json="$1"

name=$(echo $json | jq -r '.name')
age=$(echo $json | jq -r '.age')
email=$(echo $json | jq -r '.email')

echo $name
echo $age
echo $email

Command:

./script.sh '{"name": "John Doe", "age": 30, "email": "john.doe@example.com"}'

Output:

John Doe
30
john.doe@example.com

In this example, we pass the JSON data as a command-line argument to the Bash script and assign it to the json variable. Then, we use echo to pass the JSON data to jq for parsing and extract the desired fields.

Related Article: How to Manipulate Quotes & Strings in Bash Scripts

Libraries for Parsing JSON in Bash

While jq is a useful and widely-used tool for parsing JSON in Bash, there are other libraries available that provide additional functionality and flexibility. Here are a few notable libraries for parsing JSON in Bash:

1. JSON.sh

JSON.sh is a lightweight and portable shell script for parsing JSON data. It is implemented in pure Bash and does not rely on external tools or libraries. JSON.sh uses a simple and intuitive syntax to navigate and extract data from JSON objects and arrays.

Example usage:

#!/bin/bash

source json.sh

json='{
  "name": "John Doe",
  "age": 30,
  "email": "john.doe@example.com"
}'

name=$(echo $json | parse_json | extract_value name)
age=$(echo $json | parse_json | extract_value age)
email=$(echo $json | parse_json | extract_value email)

echo $name
echo $age
echo $email

Output:

John Doe
30
john.doe@example.com

In this example, we use the parse_json function to parse the JSON data and the extract_value function to extract the desired fields.

2. bash-json

bash-json is a Bash library that provides functions for parsing and manipulating JSON data. It is implemented as a single Bash script and does not require any external dependencies. bash-json offers a simple and intuitive API for working with JSON objects and arrays.

Example usage:

#!/bin/bash

source bash-json.sh

json='{
  "name": "John Doe",
  "age": 30,
  "email": "john.doe@example.com"
}'

name=$(json_get_value "$json" "name")
age=$(json_get_value "$json" "age")
email=$(json_get_value "$json" "email")

echo $name
echo $age
echo $email

Output:

John Doe
30
john.doe@example.com

In this example, we use the json_get_value function to extract the desired fields from the JSON data.

Related Article: Appending Dates to Filenames in Bash Scripts

Using jq to Import JSON Data into a Bash Script

jq is a useful and versatile command-line JSON processor that can be used to import JSON data into a Bash script. It provides a rich set of features for filtering, transforming, and manipulating JSON data, making it a popular choice for working with JSON in Bash.

Here’s an example of how to use jq to import JSON data into a Bash script:

#!/bin/bash

json=$(curl -s https://api.example.com/data)
name=$(echo $json | jq -r '.name')
age=$(echo $json | jq -r '.age')
email=$(echo $json | jq -r '.email')

echo $name
echo $age
echo $email

In this example, we use curl to fetch JSON data from an API endpoint and store it in the json variable. Then, we use jq to extract the desired fields from the JSON data and assign them to variables.

Extracting Specific Values from a JSON File in Bash

Extracting specific values from a JSON file in Bash can be easily done using jq. jq provides a wide range of features for querying and filtering JSON data, allowing you to extract specific values based on various conditions.

Here are a few examples of how to extract specific values from a JSON file using jq:

Example 1: Extracting a Single Value

Let’s say we have a JSON file named data.json with the following content:

{
  "name": "John Doe",
  "age": 30,
  "email": "john.doe@example.com",
  "address": {
    "street": "123 Main St",
    "city": "New York",
    "state": "NY"
  }
}

To extract the value of the name field from this JSON file, we can use the following command:

name=$(jq -r '.name' data.json)
echo $name

Output:

John Doe

In this example, we use the -r option to output the result as raw text, without quotes. The .name in the jq command represents the path to the name field.

Related Article: Executing a Bash Script with Multivariables in Linux

Example 2: Extracting Values from Nested Objects

If the value you want to extract is nested inside an object, you can use dot notation to specify the path to the desired field. Let’s consider the same JSON file as before:

{
  "name": "John Doe",
  "age": 30,
  "email": "john.doe@example.com",
  "address": {
    "street": "123 Main St",
    "city": "New York",
    "state": "NY"
  }
}

To extract the value of the street field from the address object, we can use the following command:

street=$(jq -r '.address.street' data.json)
echo $street

Output:

123 Main St

In this example, we use the .address.street path in the jq command to navigate to the street field inside the address object.

Syntax for Importing JSON into a Bash Script

To import JSON data into a Bash script, you can use a combination of command substitution and jq to parse and extract the desired fields. Here’s a syntax example:

#!/bin/bash

# Read JSON data from a file
json=$(cat data.json)

# Extract specific values using jq
name=$(echo $json | jq -r '.name')
age=$(echo $json | jq -r '.age')
email=$(echo $json | jq -r '.email')

# Use the extracted values in your script
echo "Name: $name"
echo "Age: $age"
echo "Email: $email"

In this example, we use the cat command to read the content of the data.json file and store it in the json variable. Then, we use echo to pass the JSON data to jq for parsing and extract the desired fields. Finally, we use the extracted values in our script as needed.

Limitations when Importing JSON Data into a Bash Script

When importing JSON data into a Bash script, there are a few limitations to be aware of:

Related Article: Making Bash Scripts Executable with Chmod in Linux

Complex JSON Structures

Bash is not designed to handle complex JSON structures, such as nested arrays or deeply nested objects. While tools like jq provide useful features for navigating and manipulating JSON data, Bash itself has limited capabilities in this regard. If you need to work with complex JSON structures, it is recommended to use a more suitable programming language like Python or JavaScript.

Performance Considerations

Parsing and processing JSON data in Bash can be relatively slow compared to other programming languages. Bash scripts are interpreted, and processing JSON data requires invoking external commands like jq. If you are working with large JSON files or need to process JSON data frequently, consider using a more performant language for improved efficiency.

JSON Validation

Bash does not provide built-in support for JSON validation. If you are importing JSON data into a Bash script, it is assumed that the JSON data is valid and well-formed. If you need to validate the JSON data before processing it, you can use external tools or libraries like jq or jsonlint to ensure the data is valid.

Related Article: Formatting and Displaying Dates with Bash Scripts in Linux

Dependency on External Tools

Importing JSON data into a Bash script often requires external tools or libraries like jq. While these tools are widely available and easy to use, they introduce a dependency on external dependencies. Make sure to document the required dependencies and provide clear instructions for installing them to ensure the script can be executed successfully on different systems.

Additional Resources

– Working with JSON in Bash using jq
– Parsing JSON with jq in Bash
– Reading JSON from Bash Script

More Articles from the The Shell Scripting Guide: From Basics to Advanced Concepts series:

How to Make a Bash Script Continue to Run After an Error

Bash scripts are a powerful tool for automating tasks in Linux systems. However, when errors occur, scripts often stop running, causing delays and inefficiencies. In... read more

How to Check the Success of a Bash Script

Learn how to check the success of a bash script execution in a Linux environment. Dive into topics such as bash script success check, error handling, determining if a... read more

Locating and Moving Files in Bash Scripting on Linux

Learn how to locate and move files in Linux using bash scripting. This article covers file search, manipulation, handling, and navigation techniques, as well as ways to... read more

How to Extract Numbers from Strings in Bash

Learn how to extract numbers from strings using bash scripting on Linux. This article covers various techniques such as regular expressions, awk, cut, tr command, expr... read more

How to Replace a Substring in a String in a Shell Script

Replacing substrings in a string within a shell script can be easily achieved using the sed command or parameter expansion. This article provides simple and practical... read more

How to Choose the Preferred Bash Shebang in Linux

Choosing the right bash shebang in the Linux environment can greatly impact the performance and compatibility of your scripts. This simple guide presents two options for... read more