I/O redirection to Linux

I/O redirection to Linux

In this lesson we will learn about the coolest feature of the command line: redirecting Input/Output. With this feature we will be able to redirect the input and output of commands from and to files, as well as compose whole pipelines from commands. To demonstrate this feature, we will enter the following commands:

cat – combines files.



sort – sort the lines of text.

uniq – reports about repeated lines or removes them.

wc – outputs the number of characters to translate strings, words and bytes in each specified file.



grep – finds and outputs the lines corresponding to the template.

head – outputs the first strings of the file.

tail – outputs the last rows from the file.

head – reads data from standard input and writes to standard output and to files.

Standard input, output and error output

.
Many of the programs we’ve used before output something to the console. This output is often divided into two types. The first type is the results of the program’s work, i.e. the data for which the program was created. The second is status messages or errors informing us about the program’s well-being. For example, if you look at the output of the ls program, you may see that it displays the results of its work and sometimes reports about errors.

According to the central idea of Unix that “everything is a file”, programs like ls actually output their results in a special file called standard output (standard output, often referred to as stdout), and status messages in a special file called standard output (standard error, stderr). By default, both files, the standard output and the standard error output, are linked to the screen and are not stored on disk.

In addition, many programs accept input from a special file called standard input (standard input, stdin), which by default is linked to the keyboard.

The Input/Output redirection mechanism allows you to change the output and input direction. Usually the output is on the screen and input is from the keyboard, but the I/O redirection mechanism allows you to change this order of things.

Referral of standard output

.
The mechanism of redirection of input/output allows you to explicitly specify where the standard output should be performed. To redirect the standard output to another file instead of the screen, you need to add the &gt redirection operator in the command; and the file name. Where can it come in handy? Sometimes it is useful to save the output of the command to a file. For example, you can tell the shell that it should forward the output of the ls command to a ls-output.txt file instead of a screen:

[[email protected] ~]$ ls -l /usr/bin > ls-output.txt

Here we created a long list of file contents /usr/bin and sent the results to the file ls-output.txt. Let’s examine the redirected output of the command:

[[email protected] ~]$ ls -l ls-output.txt

.

-rw-rw-r– 1 me me 167878 2012-02-01 15:07 ls-output.txt

It’s not a bad file. If you output the content of ls-output.txt using the command less, you can see that it really contains the results of ls:

[[email protected] ~]$ less ls-output.txt

.
Let us now repeat the redirection experiment, but with a little complication: let us specify the name of a non-existent directory:

[[email protected] ~]$ ls -l /bin/usr > ls-output.txt

ls: You cannot access ‘/bin/usr’: No such file or directory

.
We got an error message. Everything is logical – we have specified a non-existent directory /bin/usr, but why did the message appear on the screen instead of being redirected to the file ls-output.txt? The matter is that the program ls does not output error messages into the standard output. Like many decent programs for Unix, it outputs error messages into the standard error output stream. Since we have redirected only the standard output, but not the standard output, the error message appears on the screen. How to redirect the standard error output will be shown below, but before we do that let’s see what happened to our file:

[[email protected] ~]$ ls -l ls-output.txt

-rw-rw-r– 1 me 0 2012-02-01 15:08 ls-output.txt

The file’s been cleared! This is explained by the fact that when redirecting the output using the > operator, the destination file is always overwritten from the very beginning. Since the ls command did not produce any results, but only an error message, the redirection operator overwrites the file and then stops due to an error, which led to its clearing. It turns out that if you need to clear some file ( or create a new, empty file), you can do it using the following trick:

[[email protected] ~]$ > ls-output.txt

A simple redirection operator, without the command preceding it, will clear an existing file or create a new, empty file.

So how do you add output to the end of an existing file without losing it? For this purpose, we use the redirection operator >:

[[email protected] ~]$ ls -l /usr/bin >> ls-output.txt

.
The >> operator will simply add the results to the end of the file. If the file does not exist,

it will be created as when using the > operator. Let’s test it:

We repeated the command three times and received a file three times larger.

Referral of standard output-errors

Redirection of the standard error output is not as easy as the standard output. To redirect the standard error output, you need to specify its file descriptor. The program can output in any of several numbered file streams. The first three of them we mentioned as standard input, output and error output. The command shell refers to them as file descriptors 0, 1 and 2 respectively. The command shell supports file redirection syntax using file descriptor numbers. Since 2 file descriptor corresponds to the standard error output, we can redirect it as shown below:

[[email protected] ~]$ ls -l /bin/usr 2> ls-error.txt

The file descriptor number 2 is placed just before the redirection operator to redirect the standard error output to the file ls-error.txt.

Referral of standard output and standard output into one file

.
Sometimes it is necessary to save the whole output of the command in one file. To do this, redirect two threads at once, the standard output and the standard error output. There are two ways to do this. The first – traditional – works in old versions of the command shell:

[[email protected] ~]$ ls -l /bin/usr > ls-output.txt 2>&1

.
Two redirections are being made here. First – redirection of the standard output to the file ls-output.txt, and then, using notation 2>&1, – redirection of the file descriptor 2 (standard output of errors) to the file descriptor 1 (standard output).

Modern versions of bash support a second, simpler method of redirecting this type:

[[email protected] ~]$ ls -l /bin/usr &> ls-output.txt

In this example the only &> operator is used, which redirects the standard output and the standard output to the file ls-output.txt.

Remove unwanted output

.
Sometimes silence is really gold, and the team’s withdrawal must be discarded. This is especially true for service and error messages. The system allows this by providing a special file /dev/null where the output can be redirected. This file represents a system device called a bit receiver (bit bucket), or trash can, which accepts any input and does nothing with it. To suppress the output of error messages, just do the following:

[[email protected] ~]$ ls -l /bin/usr 2> /dev/null

.

Referral of standard input

So far we haven’t met any teams that use the standard input (they have actually met, but we’ll discuss them in more detail below), so we need to get to know them.

cat – file merge

The cat command reads the content of one or more files and copies it to the standard output:

cat [file…]

Often the cat command can be considered an analogue of TYPE in DOS. It is used to output the content of files without the possibility of page view. For example,

[[email protected] ~]$ cat ls-output.txt

will print the content of the file ls-output.txt. The cat command is often used to output short text files. Since cat is capable of receiving several files at once, it is used to combine them. Imagine that you have downloaded a large file that is broken down into many parts (in Usenet media files are often broken down this way) and you want to merge them into a single file. If the files have names, such as

movie.mpeg.001 movie.mpeg.002 … movie.mpeg.099

they can be combined by the next team:

[[email protected] ~]$ cat movie.mpeg.0* > movie.mpeg

Since the substitution of actual names for group symbols is always done in sort order, the arguments will be in the right order.

All this is fine, but what does standard input have to do with it? It has nothing to do with it yet, but let’s try something else. What happens if you call a cat without arguments?

[[email protected] ~]$ cat

Nothing happened – it feels like the team hung up. However, in reality, the team is doing exactly what it was supposed to do.

If you call cat without arguments, it will start reading data from the default input, and since the default input is connected to the keyboard, you get the command waiting for you to type something!

Try this:

[[email protected] ~]$ cat

Eat some more of those soft French rolls, have some tea.
Then press the CTRL-D combination (it is, holding down CTRL, press D) to tell cat that the end of the file (end-of-file, EOF) is reached in the standard input:

[[email protected] ~]$ cat

Eat some more of those soft French rolls, have some tea.

Eat some more of those soft French rolls, have some tea.
In the absence of arguments with file names cat copies the content of the standard input into the standard output, that’s why we saw it repeat the line we entered. This feature can be used to create short text files. Imagine that you had to create a file named eat_more.txt containing text from the example above. You could have done it this way:

[[email protected] ~]$ cat > eat_more.txt

Eat some more of those soft French rolls, have some tea.
Enter the command, then the text you want to put into the file, and don’t forget to click the CTRL-D combination at the end. Using the command line, we have implemented the world’s easiest word processor! To see the result, we will use the cat command and copy the file to the standard output:

[[email protected] ~]$ cat eat_more.txt

Eat some more of those soft French rolls, have some tea.
Now that we know that the cat command can accept data not only from the files specified in the arguments but also from the standard input, let’s try to redirect the standard input:

[[email protected] ~]$ cat < eat_more.txt

Eat some more of those soft French rolls, have some tea.
Using the redirection operator <, we changed the data source for standard input from the keyboard to the file eat_more.txt. As you can see, the result is the same as if we just passed a single argument with the file name. This method has no advantages over passing a simple argument, but it demonstrates how you can use files as a data source for standard input. Other commands find a better use for standard input, which we will soon see.

Before you move on, read the help page (man) for cat as it has some very interesting parameters.



WARNING! All links in the articles may lead to malicious sites or contain viruses. Follow them at your own risk. Those who purposely visit the article know what they are doing. Do not click on everything thoughtlessly.


46 Views

0 0 vote
Article Rating
Subscribe
Notify of
guest
0 Comments
Inline Feedbacks
View all comments


Do NOT follow this link or you will be banned from the site!
0
Would love your thoughts, please comment.x
()
x

Spelling error report

The following text will be sent to our editors: