User Tools

Site Tools


labs:bash

This is an old revision of the document!


Bash & Git (private)

In this lab session, you're supposed to learn the basics of bash and git.

Bash is the standard shell in many Linux distributions. Despite the ancient feeling, some tasks which are very tedious to do on a GUI can be solved pretty quickly on the command line. Especially if a task is very repetitive, there is a high chance that there exists a command line tool which can solve it faster.

Git is a version control system. Version control systems are primarily used in software development to keep track of all changes made by the software developers to any file within the software project. It keeps a version history and allows you to revert changes made to a file and to restore deleted files. Therefore, git is also useful if you work on a set of files on your own, but you want to keep track of every change you make and you want to be able to go back to a previous version if you make a mistake.

Useful resources

Exercise 1: Bash and Git basics

In this exercise, you will use the most basic commands of git and bash. Make yourself familiar with these tools before we start with the more interesting part in Exercise 2.

  1. Clone the git repository 10.0.0.1/opt/git/ti2fp into a folder called ti2fp. The username is git and the password is ti2lab.
  2. Create a new folder with the name of your nethz account.
  3. Navigate into this folder and create a new git repository.
  4. Copy the files and subfolders from the ti2fp folder into this folder.
  5. Add these files to the git Index and commit them.
  6. If you have worked with the command line before, you can skip the rest of this exercise and proceed to Exercise 2.
  7. We strongly recommend that, in the following exercises, you commit your changes every time you solve an exercise. This will allow you to undo any mistakes that you make.
  8. Create a new file called empty.txt.
  9. List all files in that folder.
  10. Create a copy of the file empty.txt called still-empty.txt.
  11. Open the file empty.txt in the text editor called nano, add some text and save the file.
  12. Output the content of the file on the command line.
  13. The file is not empty anymore, so let's rename it to text.txt.
  14. Delete the file still-empty.txt.

Exercise 2: Extracting data from files

In this exercise, you will see some basic commands to read data from files. You can save the output of a program to a file by using redirects.

  1. Navigate into the csv-files folder. There, you will find a number of .csv files which contain stock prices of different stocks.
  2. Print the content of msft.csv on the terminal.
  3. Create a file called allTheData.csv that contains every line from every *.csv file in the main folder.
  4. Count the number of lines in all .csv files and write them into the file numberOfLines.txt.
  5. You want to have a quick look at the data format in your .csv files. Write the first 10 lines of every .csv file into the file firstLines.txt.
  6. Your friend is interested in the latest stock prices. Write the last 20 lines of every .csv file into the file latestData.txt.
  7. Write the first three columns (Date, Open, High) of msft.csv into msft2.csv.

Exercise 3: Searching, pipes and xargs

In the previous exercise, all tasks could be solved with just one command by passing some parameters to this command. In this exercise, we will learn how to search for files by their name and their content. Moreover, you are supposed to use Pipes and (sometimes) the xargs command to pass on the output of one command to another command.

Go to the sourcecode folder. There, you will find hundreds of Java files in different subfolders. It would be tedious to open them all in a text editor if you were looking for something, wouldn't it?

  1. Write the name of every .java file (including the path to the files starting from the folder sourcecode) into a file called javaFilesList.txt.
  2. Print every line that contains the string author or Author.
  3. List the name of every file that does not contain the string java in its filename. Write the output into the file otherFilesList.txt.
  4. Print the filename of every file that contains the string // Author: University of Zurich.
  5. Delete those files.
  6. Commit your changes. Make sure that git also commits the removal of those files.
  7. Go to the folder sourcecode/utils. Print every file in this folder that does not contain DO NOT DELETE.
  8. Delete those files.
  9. You realize that you did not want to delete these files. Use Git to restore the files that you removed in Exercise 8.
    • This reverts all files in the working tree to the current HEAD revision.
    • This reverts all unstaged changes in the working tree, including deleted files. If the deleted files have already been staged for commit, one would need to do git reset HEAD . first.
    • Both these methods discard any changes that have not yet been commited!!!
    • Here are two very neat solutions from Stackoverflow which only affect deleted files, but do not revert any other unstaged changes:
      1. To recover all unstaged deletions at once, automatically, without specifying each single path:
      2. To recover all staged deletions at once, automatically, without specifying each single path:
  10. Use Git to restore the files that you removed in exercise 5.
    There's at least two possible solution to this exercise:
    • The best solution would probably be to use git log to find the hash of the commit that was made after removing the files and to revert that commit using
    • Another possibility would be to do a git reset. This just discards the previous commit. It is dangerous to do this if the commit has already been pushed to a server and might have been checked out by other people, because then you might run into an inconsistent state. Anyway, it works like this: where <hash> is the hash of the commit you want to go back to, i.e. the last commit before the files were deleted.
  11. The file allSrc.txt (in the folder sourcecode) contains a list of files. Write every line of these files that contain the word static into the file static.txt.

Exercise 4: sed and Git branches

sed is a very powerful tool to edit text files on the command line. It is often used for search-and-replace operations. As you will see shortly, you can do a simple search-and-replace operation or delete a couple of lines - all of this can be done in many files at once.

You will also see Git Branches in this exercise.

  1. Create a git branch called experimental.
  2. Check out your newly created branch.
  3. Go to the folder core.
  4. Replace the string // Author: University of Zurich with // Author: ETH Zurich in all files in this directory.
  5. Navigate to the csv-files directory. Remove every odd line from every csv file in this folder.
  6. Go back to the Git master branch.

Exercise 5: Bash scripting

In this exercise we advise you to write bash scripts, i.e., *.sh files. They will make life a bit easier. You can find some basic information here and here.

  1. You work together with a colleague who uses Windows. Thus, he cannot checkout the repository (because the files in the folder files with dots contain a :). Replace all the : with _.
  2. Write a script that prints out is running if looper is running and is not running otherwise. Note that this process is currently not running.
  3. Update your script such that it starts looper if it is not running.
  4. Update your script such that it continuously monitors looper.
  5. Update your script such that it accepts the names of several programs as parameter, i.e., ./yourScript.sh looper backGround should work.
labs/bash.1487336698.txt.gz · Last modified: 2020/08/31 21:03 (external edit)