Knowledge is power

SIMPLE FILTERS

2010-11-29T01:18:00.000-08:00

Filters are the commands which accept data from standard input manipulate it and write the results to standard output. Filters are the central tools of the UNIX tool kit, and each filter performs a simple function. Some commands use delimiter, pipe (|) or colon (:). Many filters work well with delimited fields, and some simply won’t work without them. The piping mechanism allows the standard output of one filter serve as standard input of another. The filters can read data from standard input when used without a filename as argument, and from the file otherwise

The Simple Database

Several UNIX commands are provided for text editing and shell programming. (emp.lst) - each line of this file has six fields separated by five delimiters. The details of an employee are stored in one single line. This text file designed in fixed format and containing a personnel database. There are 15 lines, where each field is separated by the delimiter |.

$ cat emp.lst

2233 | a.k.shukla | g.m | sales | 12/12/52 | 6000

9876 | jai sharma | director | production | 12/03/50 | 7000

5678 | sumit chakrobarty | d.g.m. | marketing | 19/04/43 | 6000

2365 | barun sengupta | director | personnel | 11/05/47 | 7800

5423 | n.k.gupta | chairman | admin | 30/08/56 | 5400

1006 | chanchal singhvi | director | sales | 03/09/38 | 6700

6213 | karuna ganguly | g.m. | accounts | 05/06/62 | 6300

1265 | s.n. dasgupta | manager | sales | 12/09/63 | 5600

4290 | jayant choudhury | executive | production | 07/09/50 | 6000

2476 | anil aggarwal | manager | sales | 01/05/59 | 5000

6521 | lalit chowdury | directir | marketing | 26/09/45 | 8200

3212 | shyam saksena | d.g.m. | accounts | 12/12/55 | 6000

3564 | sudhir agarwal | executive | personnel | 06/07/47 | 7500

2345 | j. b. sexena | g.m. | marketing | 12/03/45 | 8000

0110 | v.k.agrawal | g.m.| marketing | 31/12/40 | 9000

pr : paginating files

We know that,

cat dept.lst

01|accounts|6213

02|progs|5423

03|marketing|6521

04|personnel|2365

05|production|9876

06|sales|1006

pr command adds suitable headers, footers and formatted text. pr adds five lines of margin at the top and bottom. The header shows the date and time of last modification of the file along with the filename and page number.

pr dept.lst

May 06 10:38 1997 dept.lst page 1

01:accounts:6213

02:progs:5423

03:marketing:6521

04:personnel:2365

05:production:9876

06:sales:1006

…blank lines…

pr options

The different options for pr command are:

-k prints k (integer) columns

-t to suppress the header and footer

-h to have a header of user’s choice

-d double spaces input

-n will number each line and helps in debugging

-on offsets the lines by n spaces and increases left margin of page

pr +10 chap01

starts printing from page 10

pr -l 54 chap01

this option sets the page length to 54

head – displaying the beginning of the file

The command displays the top of the file. It displays the first 10 lines of the file, when used without an option.

head emp.lst

-n to specify a line count

head -n 3 emp.lst

will display the first three lines of the file.

tail – displaying the end of a file

This command displays the end of the file. It displays the last 10 lines of the file, when used without an option.

tail emp.lst

-n to specify a line count

tail -n 3 emp.lst

displays the last three lines of the file. We can also address lines from the beginning of the file instead of the end. The +count option allows to do that, where count represents the line number from where the selection should begin.

tail +11 emp.lst

Will display 11^th line onwards

Different options for tail are:

Monitoring the file growth (-f)
Extracting bytes rather than lines (-c)

Use tail –f when we are running a program that continuously writes to a file, and we want to see how the file is growing. We have to terminate this command with the interrupt key.

cut – slitting a file vertically

It is used for slitting the file vertically. head -n 5 emp.lst | tee shortlist will select the first five lines of emp.lst and saves it to shortlist. We can cut by using -c option with a list of column numbers, delimited by a comma (cutting columns).

cut -c 6-22,24-32 shortlist

cut -c -3,6-22,28-34,55- shortlist

The expression 55- indicates column number 55 to end of line. Similarly, -3 is the same as 1-3.

Most files don’t contain fixed length lines, so we have to cut fields rather than columns (cutting fields).

-d for the field delimiter

-f for the field list

cut -d \ | -f 2,3 shortlist | tee cutlist1

will display the second and third columns of shortlist and saves the output in cutlist1. here | is escaped to prevent it as pipeline character

• To print the remaining fields, we have

cut –d \ | -f 1,4- shortlist > cutlist2

paste – pasting files

When we cut with cut, it can be pasted back with the paste command, vertically rather than horizontally. We can view two files side by side by pasting them. In the previous topic, cut was used to create the two files cutlist1 and cutlist2 containing two cut-out portions of the same file.

paste cutlist1 cutlist2

We can specify one or more delimiters with -d

paste -d “|” cutlist1 cutlist2

Where each field will be separated by the delimiter |. Even though paste uses at least two files for concatenating lines, the data for one file can be supplied through the standard input.

Joining lines (-s)

Let us consider that the file address book contains the details of three persons

cat addressbook

paste -s addressbook -to print in one single line

paste -s -d ”| | \n” addressbook -are used in a circular manner

sort : ordering a file

Sorting is the ordering of data in ascending or descending sequence. The sort command orders a file and by default, the entire line is sorted

sort shortlist

This default sorting sequence can be altered by using certain options. We can also sort one or more keys (fileds) or use a different ordering rule.

sort options

The important sort options are:

-tchar uses delimiter char to identify fields

-k n sorts on nth field

-k m,n starts sort on mth field and ends sort on nth field

-k m.n starts sort on nth column of mth field

-u removes repeated lines

-n sorts numerically

-r reverses sort order

-f folds lowercase to equivalent uppercase

-m list merges sorted files in list

-c checks if file is sorted

-o flname places output in file flname

sort –t“|” –k 2 shortlist

sorts the second field (name)

sort –t”|” –r –k 2 shortlist or

sort –t”|” –k 2r shortlist

sort order can be revered with this –r option.

sort –t”|” –k 3,3 –k 2,2 shortlist

sorting on secondary key is also possible as shown above.

sort –t”|” –k 5.7,5.8 shortlist

we can also specify a character position with in a field to be the beginning of sort as shown above (sorting on columns).

sort –n numfile

when sort acts on numericals, strange things can happen. When we sort a file containing only numbers, we get a curious result. This can be overridden by –n (numeric) option.

cut –d “|” –f3 emp.lst | sort –u | tee desigx.lst

Removing repeated lines can be possible using –u option as shown above. If we cut out the designation filed from emp.lst, we can pipe it to sort to find out the unique designations that occur in the file.

Other sort options are:

sort –o sortedlist –k 3 shortlist

sort –o shortlist shortlist

sort –c shortlist

sort –t “|” –c –k 2 shortlist

sort –m foo1 foo2 foo3

uniq command – locate repeated and nonrepeated lines

When we concatenate or merge files, we will face the problem of duplicate entries creeping in. we saw how sort removes them with the –u option. UNIX offers a special tool to handle these lines – the uniq command. Consider a sorted dept.lst that includes repeated lines:

cat dept.lst

displays all lines with duplicates. Where as,

uniq dept.lst

simply fetches one copy of each line and writes it to the standard output. Since uniq requires a sorted file as input, the general procedure is to sort a file and pipe its output to uniq. The following pipeline also produces the same output, except that the output is saved in a file:

sort dept.lst | uniq – uniqlist

Different uniq options are :

Selecting the nonrepeated lines (-u)

cut –d “|” –f3 emp.lst | sort | uniq –u

Selecting the duplicate lines (-d)

cut –d “|” –f3 emp.lst | sort | uniq –d

Counting frequency of occurrence (-c)

cut –d “|” –f3 emp.lst | sort | uniq –c

tr command – translating characters

The tr filter manipulates the individual characters in a line. It translates characters using one or two compact expressions.

tr options expn1 expn2 standard input

It takes input only from standard input, it doesn’t take a filename as argument. By default, it translates each character in expression1 to its mapped counterpart in expression2. The first character in the first expression is replaced with the first character in the second expression, and similarly for the other characters.

tr ‘|/’ ‘~-’ < emp.lst | head –n 3

exp1=‘|/’ ; exp2=‘~-’

tr “$exp1” “$exp2” < emp.lst

Changing case of text is possible from lower to upper for first three lines of the file.

head –n 3 emp.lst | tr ‘[a-z]’ ‘[A-Z]’

Different tr options are:

Deleting charecters (-d)

tr –d ‘|/’ < emp.lst | head –n 3

Compressing multiple consecutive charecters (-s)

tr –s ‘ ‘ < emp.lst | head –n 3

Complementing values of expression (-c)

tr –cd ‘|/’ < emp.lst

Using ASCII octal values and escape sequences

tr ‘|’ ‘\012’ < emp.lst | head –n 6

UNIX System Calls

2010-11-26T02:49:00.000-08:00

A system call is just what its name implies -- a request for the

operating system to do something on behalf of the user's program. The

system calls are functions used in the kernel itself. To the

programmer, the system call appears as a normal C function call.

However since a system call executes code in the kernel, there must be a

mechanism to change the mode of a process from user mode to kernel mode.

The C compiler uses a predefined library of functions (the C library)

that have the names of the system calls. The library functions

typically invoke an instruction that changes the process execution mode

to kernel mode and causes the kernel to start executing code for system

calls. The instruction that causes the mode change is often referred to

as an "operating system trap" which is a software generated interrupt.

The library routines execute in user mode, but the system call interface

is a special case of an interrupt handler. The library functions pass

the kernel a unique number per system call in a machine dependent way --

either as a parameter to the operating system trap, in a particular

system call the user is invoking. In handling the operating system

trap, the kernel looks up the system call number in a table to find the

address of the appropriate kernel routine that is the entry point for

the system call and to find the number of parameters the system call

expects. The kernel calculates the (user) address of the first

parameter to the system call by adding (or subtracting, depending on the

direction of stack growth) an offset to the user stack pointer,

corresponding to the number of the parameters to the system call.

Finally, it copies the user parameters to the "u area" and call the

appropriate system call routine. After executing the code for the

system call, the kernel determines whether there was an error. If so,

it adjusts register locations in the saved user register context,

typically setting the "carry" bit for the PS (processor status) register

and copying the error number into register 0 location. If there were no

errors in the execution of the system call, the kernel clears the

"carry" bit in the PS register and copies the appropriate return values

from the system call into the locations for registers 0 and 1 in the

saved user register context. When the kernel returns from the operating

system trap to user mode, it returns to the library instruction after

the trap instruction. The library interprets the return values from the

kernel and returns a value to the user program.

UNIX system calls are used to manage the file system, control processes,

and to provide interprocess communication. The UNIX system interface

consists of about 80 system calls (as UNIX evolves this number will

increase). The following table lists about 40 of the more important

system call:

GENERAL CLASS SPECIFIC CLASS SYSTEM CALL

------------------------------------------------------------------------------------

File Structure Creating a Channel creat()

Related Calls open()

close()

Input/Output read()

write()

Random Access lseek()

Channel Duplication dup()

Aliasing and Removing link()

Files unlink()

File Status stat()

fstat()

Access Control access()

chmod()

chown()

umask()

Device Control ioctl()

---------------------------------------------------------------------

Process Related Process Creation and exec()

Calls Termination fork()

wait()

exit()

Process Owner and Group getuid()

geteuid()

getgid()

getegid()

Process Identity getpid()

getppid()

Process Control signal()

kill()

alarm()

Change Working Directory chdir()

----------------------------------------------------------------------

Interprocess Pipelines pipe()

Communication Messages msgget()

msgsnd()

msgrcv()

msgctl()

Semaphores semget()

semop()

Shared Memory shmget()

shmat()

shmdt()

----------------------------------------------------------------------

[NOTE: The system call interface is that aspect of UNIX that has

changed the most since the inception of the UNIX system. Therefore,

when you write a software tool, you should protect that tool by putting

system calls in other subroutines within your program and then calling

only those subroutines. Should the next version of the UNIX system

change the syntax and semantics of the system calls you've used, you

need only change your interface routines.]

When a system call discovers and error, it returns -1 and stores the

reason the called failed in an external variable named "errno". The

"/usr/include/errno.h" file maps these error numbers to manifest

constants, and it these constants that you should use in your programs.

When a system call returns successfully, it returns something other than

-1, but it does not clear "errno". "errno" only has meaning directly

after a system call that returns an error.

When you use system calls in your programs, you should check the value

returned by those system calls. Furthermore, when a system call

discovers an error, you should use the "perror()" subroutine to print a

diagnostic message on the standard error file that describes why the

system call failed. The syntax for "perror()" is:

void perror(string)

char string;

"perror()" displays the argument string, a colon, and then the error

message, as directed by "errno", followed by a newline. The output of

"perror()" is displayed on "standard error". Typically, the argument

give to "perror()" is the name of the program that incurred the error,

argv[0]. However, when using subroutines and system calls on files, the

related file name might be passed to "perror()".

There are occasions where you the programmer might wish to maintain more

control over the printing of error messages than "perror()" provides --

such as with a formatted screen where the newline printed by "perror()"

would destroy the formatting. In this case, you can directly access the

same system external (global) variables that "perror()" uses. They are:

extern int errno;

extern char *sys_errlist[];

extern int sys_nerr;

"errno" has been described above. "sys_errlist" is an array (table) of

pointers to the error message strings. Each message string is null

terminated and does not contain a newline. "sys_nerr" is the number of

messages in the error message table and is the maximum value "errno" can

assume. "errno" is used as the index into the table of error messages.

Following are two sample programs that display all of the system error

messages on standard error.

/* errmsg1.c

print all system error messages using "perror()"

#include

int main()

{

int i;

extern int errno, sys_nerr;

for (i = 0; i < sys_nerr; ++i)

{

fprintf(stderr, "%3d",i);

errno = i;

perror(" ");

}

exit (0);

}

/* errmsg2.c

print all system error messages using the global error message table.

#include

int main()

{

int i;

extern int sys_nerr;

extern char *sys_errlist[];

fprintf(stderr,"Here are the current %d error messages:\n\n",sys_nerr);

for (i = 0; i < sys_nerr; ++i)

fprintf(stderr,"%3d: %s\n", i, sys_errlist[i]);

}

Following are some examples in the use of the most often used system

calls.

File Structure Related System Calls

The file structure related system calls available in the UNIX system let

you create, open, and close files, read and write files, randomly access

files, alias and remove files, get information about files, check the

accessibility of files, change protections, owner, and group of files,

and control devices. These operations either use a character string

that defines the absolute or relative path name of a file, or a small

integer called a file descriptor that identifies the I/O channel. A

channel is a connection between a process and a file that appears to the

process as an unformatted stream of bytes. The kernel presents and

accepts data from the channel as a process reads and writes that

channel. To a process then, all input and output operations are

synchronous and unbuffered.

When doing I/O, a process specifies the file descriptor for an I/O

channel, a buffer to be filled or emptied, and the maximum size of data

to be transferred. An I/O channel may allow input, output, or both.

Furthermore, each channel has a read/write pointer. Each I/O operation

starts where the last operation finished and advances the pointer by the

number of bytes transferred. A process can access a channel's data

randomly by changing the read/write pointer.

All input and output operations start by opening a file using either the

"creat()" or "open()" system calls. These calls return a file

descriptor that identifies the I/O channel. Recall that file

descriptors 0, 1, and 2 refer to standard input, standard output, and

standard error files respectively, and that file descriptor 0 is a

channel to your terminal's keyboard and file descriptors 1 and 2 are

channels to your terminal's display screen.

creat()

The prototype for the creat() system call is:

int creat(file_name, mode)

char *file_name;

int mode;

where file_name is pointer to a null terminated character string that

names the file and mode defines the file's access permissions. The mode

is usually specified as an octal number such as 0666 that would mean

read/write permission for owner, group, and others or the mode may also

be entered using manifest constants defined in the "/usr/include/sys/stat.h"

file. If the file named by file_name does not exist, the UNIX system creates

it with the specified mode permissions. However, if the file does exist, its

contents are discarded and the mode value is ignored. The permissions of the

existing file are retained.

Following is an example of how to use creat():

/* creat.c */

#include

#include /* defines types used by sys/stat.h */

#include /* defines S_IREAD & S_IWRITE */

int main()

{

int fd;

fd = creat("datafile.dat", S_IREAD | S_IWRITE);

if (fd == -1)

printf("Error in opening datafile.dat\n");

else

{

printf("datafile.dat opened for read/write access\n");

printf("datafile.dat is currently empty\n");

}

close(fd);

exit (0);

}

The following is a sample of the manifest constants for the mode

argument as defined in /usr/include/sys/stat.h:

#define S_IRWXU 0000700 /* -rwx------ */

#define S_IREAD 0000400 /* read permission, owner */

#define S_IRUSR S_IREAD

#define S_IWRITE 0000200 /* write permission, owner */

#define S_IWUSR S_IWRITE

#define S_IEXEC 0000100 /* execute/search permission, owner */

#define S_IXUSR S_IEXEC

#define S_IRWXG 0000070 /* ----rwx--- */

#define S_IRGRP 0000040 /* read permission, group */

#define S_IWGRP 0000020 /* write " " */

#define S_IXGRP 0000010 /* execute/search " " */

#define S_IRWXO 0000007 /* -------rwx */

#define S_IROTH 0000004 /* read permission, other */

#define S_IWOTH 0000002 /* write " " */

#define S_IXOTH 0000001 /* execute/search " " */

Multiple mode values may be combined by or'ing (using the | operator)

the values together as demonstrated in the above sample program.

open()

Next is the open() system call. open() lets you open a file for

reading, writing, or reading and writing.

The prototype for the open() system call is:

#include

int open(file_name, option_flags [, mode])

char *file_name;

int option_flags, mode;

where file_name is a pointer to the character string that names the

file, option_flags represent the type of channel, and mode defines the

file's access permissions if the file is being created.

The allowable option_flags as defined in "/usr/include/fcntl.h" are:

#define O_RDONLY 0 /* Open the file for reading only */

#define O_WRONLY 1 /* Open the file for writing only */

#define O_RDWR 2 /* Open the file for both reading and writing*/

#define O_NDELAY 04 /* Non-blocking I/O */

#define O_APPEND 010 /* append (writes guaranteed at the end) */

#define O_CREAT 00400 /*open with file create (uses third open arg) */

#define O_TRUNC 01000 /* open with truncation */

#define O_EXCL 02000 /* exclusive open */

Multiple values are combined using the | operator (i.e. bitwise OR).

Note: some combinations are mutually exclusive such as: O_RDONLY |

O_WRONLY and will cause open() to fail. If the O_CREAT flag is used,

then a mode argument is required. The mode argument may be specified in

the same manner as in the creat() system call.

Following is an example of how to use open():

/* open.c */

#include /* defines options flags */

#include /* defines types used by sys/stat.h */

#include /* defines S_IREAD & S_IWRITE */

static char message[] = "Hello, world";

int main()

{

int fd;

char buffer[80];

/* open datafile.dat for read/write access (O_RDWR)

create datafile.dat if it does not exist (O_CREAT)

return error if datafile already exists (O_EXCL)

permit read/write access to file (S_IWRITE | S_IREAD)

fd = open("datafile.dat",O_RDWR | O_CREAT | O_EXCL, S_IREAD | S_IWRITE);

if (fd != -1)

{

printf("datafile.dat opened for read/write access\n");

write(fd, message, sizeof(message));

lseek(fd, 0L, 0); /* go back to the beginning of the file */

if (read(fd, buffer, sizeof(message)) == sizeof(message))

printf("\"%s\" was written to datafile.dat\n", buffer);

else

printf("*** error reading datafile.dat ***\n");

close (fd);

}

else

printf("*** datafile.dat already exists ***\n");

exit (0);

}

close()

To close a channel, use the close() system call. The prototype for the

close() system call is:

int close(file_descriptor)

int file_descriptor;

where file_descriptor identifies a currently open channel. close()

fails if file_descriptor does not identify a currently open channel.

read() write()

The read() system call does all input and the write() system call does

all output. When used together, they provide all the tools necessary to

do input and output sequentially. When used with the lseek() system

call, they provide all the tools necessary to do input and output

randomly.

Both read() and write() take three arguments. Their prototypes are:

int read(file_descriptor, buffer_pointer, transfer_size)

int file_descriptor;

char *buffer_pointer;

unsigned transfer_size;

int write(file_descriptor, buffer_pointer, transfer_size)

int file_descriptor;

char *buffer_pointer;

unsigned transfer_size;

where file_descriptor identifies the I/O channel, buffer_pointer points

to the area in memory where the data is stored for a read() or where

the data is taken for a write(), and transfer_size defines the maximum

number of characters transferred between the file and the buffer.

read() and write() return the number of bytes transferred.

There is no limit on transfer_size, but you must make sure it's safe to

copy transfer_size bytes to or from the memory pointed to by

buffer_pointer. A transfer_size of 1 is used to transfer a byte at a

time for so-called "unbuffered" input/output. The most efficient value

for transfer_size is the size of the largest physical record the I/O

channel is likely to have to handle. Therefore, 1K bytes -- the disk

block size -- is the most efficient general-purpose buffer size for a

standard file. However, if you are writing to a terminal, the transfer

is best handled in lines ending with a newline.

For an example using read() and write(), see the above example of

open().

lseek()

The UNIX system file system treats an ordinary file as a sequence of

bytes. No internal structure is imposed on a file by the operating

system. Generally, a file is read or written sequentially -- that is,

from beginning to the end of the file. Sometimes sequential reading and

writing is not appropriate. It may be inefficient, for instance, to

read an entire file just to move to the end of the file to add

characters. Fortunately, the UNIX system lets you read and write

anywhere in the file. Known as "random access", this capability is made

possible with the lseek() system call. During file I/O, the UNIX system

uses a long integer, also called a File Pointer, to keep track of the

next byte to read or write. This long integer represents the number of

bytes from the beginning of the file to that next character. Random

access I/O is achieved by changing the value of this file pointer using

the lseek() system call.

The prototype for lseek() is:

long lseek(file_descriptor, offset, whence)

int file_descriptor;

long offset;

int whence;

where file_descriptor identifies the I/O channel and offset and whence

work together to describe how to change the file pointer according to

the following table:

whence new position

------------------------------

0 offset bytes into the file

1 current position in the file plus offset

2 current end-of-file position plus offset

If successful, lseek() returns a long integer that defines the new file

pointer value measured in bytes from the beginning of the file. If

unsuccessful, the file position does not change.

Certain devices are incapable of seeking, namely terminals and the

character interface to a tape drive. lseek() does not change the file

pointer to these devices.

Following is an example using lseek():

/* lseek.c */

#include

int main()

{

int fd;

long position;

fd = open("datafile.dat", O_RDONLY);

if ( fd != -1)

{

position = lseek(fd, 0L, 2); /* seek 0 bytes from end-of-file */

if (position != -1)

printf("The length of datafile.dat is %ld bytes.\n", position);

else

perror("lseek error");

}

else

printf("can't open datafile.dat\n");

close(fd);

}

Many UNIX systems have defined manifest constants for use as the

"whence" argument of lseek(). The definitions can be found in the

"file.h" and/or "unistd.h" include files. For example, the University

of Maryland's HP-9000 UNIX system has the following definitions:

from file.h we have:

#define L_SET 0 /* absolute offset */

#define L_INCR 1 /* relative to current offset */

#define L_XTND 2 /* relative to end of file */

and from unistd.h we have:

#define SEEK_SET 0 /* Set file pointer to "offset" */

#define SEEK_CUR 1 /* Set file pointer to current plus "offset" */

#define SEEK_END 2 /* Set file pointer to EOF plus "offset" */

The definitions from unistd.h are the most "portable" across UNIX and

MS-DOS C compilers.

link()

The UNIX system file structure allows more than one named reference to a

given file, a feature called "aliasing". Making an alias to a file

means that the file has more than one name, but all names of the file

refer to the same data. Since all names refer to the same data,

changing the contents of one file changes the contents of all aliases to

that file. Aliasing a file in the UNIX system amounts to the system

creating a new directory entry that contains the alias file name and

then copying the i-number of a existing file to the i-number position of

this new directory entry. This action is accomplished by the link()

system call. The link() system call links an existing file to a new

file.

The prototype for link() is:

int link(original_name, alias_name)

char *original_name, *alias_name;

where both original_name and alias_name are character strings that name

the existing and new files respectively. link() will fail and no link

will be created if any of the following conditions holds:

a path name component is not a directory.

a path name component does not exist.

a path name component is off-limits.

original_name does not exist.

alias_name does exist.

original_name is a directory and you are not the superuser.

a link is attempted across file systems.

the destination directory for alias_name is not writable.

the destination directory is on a mounted read-only file system.

Following is a short example:

/* link.c

#include

int main()

{

if ((link("foo.old", "foo.new")) == -1)

{

perror(" ");

exit (1); /* return a non-zero exit code on error */

}

exit(0);

}

unlink()

The opposite of the link() system call is the unlink() system call.

unlink() removes a file by zeroing the i-number part of the file's

directory entry, reducing the link count field in the file's inode by 1,

and releasing the data blocks and the inode if the link count field

becomes zero. unlink() is the only system call for removing a file in

the UNIX system.

The prototype for unlink() is:

int unlink(file_name)

char *file_name;

where file_name names the file to be unlinked. unlink() fails if any of

the following conditions holds:

a path name component is not a directory.

a path name component does not exist.

a path name component is off-limits.

file_name does not exist.

file_name is a directory and you are not the superuser.

the directory for the file named by file_name is not writable.

the directory is contained in a file system mounted read-only.

It is important to understand that a file's contents and its inode are

not discarded until all processes close the unlinked file.

Following is a short example:

/* unlink.c

#include

int main()

{

if ((unlink("foo.bar")) == -1)

{

perror(" ");

exit (1); /* return a non-zero exit code on error */

}

exit (0);

}

File Status

stat() - fstat()

The i-node data structure holds all the information about a file except the

file's name and its contents. Sometimes your programs need to use the

information in the i-node structure to do some job. You can access this

information with the stat() and fstat() system calls. stat() and fstat()

return the information in the i-node for the file named by a string and by a

file descriptor, respectively. The format for the i-node struct returned by

these system calls is defined in /usr/include/sys/stat.h. stat.h uses types

built with the C language typedef construct and defined in the file

/usr/include/sys/types.h, so it too must be included and must be included

before the inclusion of the stat.h file.

The prototypes for stat() and fstat() are:

#include

int stat(file_name, stat_buf)

char *file_name;

struct stat *stat_buf;

int fstat(file_descriptor, stat_buf)

int file_descriptor;

struct stat *stat_buf;

where file_name names the file as an ASCII string and file_descriptor names

the I/O channel and therefore the file. Both calls returns the file's

specifics in stat_buf. stat() and fstat() fail if any of the following

conditions hold:

a path name component is not a directory (stat() only).

file_name does not exit (stat() only).

a path name component is off-limits (stat() only).

file_descriptor does not identify an open I/O channel (fstat() only).

stat_buf points to an invalid address.

Following is an extract of the stat.h file from the University's HP-9000. It

shows the definition of the stat structure and some manifest constants used

to access the st_mode field of the structure.

/* stat.h */

struct stat

{

dev_t st_dev; /* The device number containing the i-node */

ino_t st_ino; /* The i-number */

unsigned short st_mode; /* The 16 bit mode */

short st_nlink; /* The link count; 0 for pipes */

ushort st_uid; /* The owner user-ID */

ushort st_gid; /* The group-ID */

dev_t st_rdev; /* For a special file, the device number */

off_t st_size; /* The size of the file; 0 for special files */

time_t st_atime; /* The access time. */

int st_spare1;

time_t st_mtime; /* The modification time. */

int st_spare2;

time_t st_ctime; /* The status-change time. */

int st_spare3;

long st_blksize;

long st_blocks;

uint st_remote:1; /* Set if file is remote */

dev_t st_netdev; /* ID of device containing */

/* network special file */

ino_t st_netino; /* Inode number of network special file */

long st_spare4[9];

};

#define S_IFMT 0170000 /* type of file */

#define S_IFDIR 0040000 /* directory */

#define S_IFCHR 0020000 /* character special */

#define S_IFBLK 0060000 /* block special */

#define S_IFREG 0100000 /* regular (ordinary) */

#define S_IFIFO 0010000 /* fifo */

#define S_IFNWK 0110000 /* network special */

#define S_IFLNK 0120000 /* symbolic link */

#define S_IFSOCK 0140000 /* socket */

#define S_ISUID 0004000 /* set user id on execution */

#define S_ISGID 0002000 /* set group id on execution */

#define S_ENFMT 0002000 /* enforced file locking (shared with S_ISGID)*/

#define S_ISVTX 0001000 /* save swapped text even after use */

Following is an example program demonstrating the use of the stat() system

call to determine the status of a file:

/* status.c */

/* demonstrates the use of the stat() system call to determine the

status of a file.

#include

#define ERR (-1)

#define TRUE 1

#define FALSE 0

int main();

int main(argc, argv)

int argc;

char *argv[];

{

int isdevice = FALSE;

struct stat stat_buf;

if (argc != 2)

{

printf("Usage: %s filename\n", argv[0]);

exit (1);

}

if ( stat( argv[1], &stat_buf) == ERR)

{

perror("stat");

exit (1);

}

printf("\nFile: %s status:\n\n",argv[1]);

if ((stat_buf.st_mode & S_IFMT) == S_IFDIR)

printf("Directory\n");

else if ((stat_buf.st_mode & S_IFMT) == S_IFBLK)

{

printf("Block special file\n");

isdevice = TRUE;

}

else if ((stat_buf.st_mode & S_IFMT) == S_IFCHR)

{

printf("Character special file\n");

isdevice = TRUE;

}

else if ((stat_buf.st_mode & S_IFMT) == S_IFREG)

printf("Ordinary file\n");

else if ((stat_buf.st_mode & S_IFMT) == S_IFIFO)

printf("FIFO\n");

-32-

if (isdevice)

printf("Device number:%d, %d\n", (stat_buf.st_rdev > 8) & 0377,

stat_buf.st_rdev & 0377);

printf("Resides on device:%d, %d\n", (stat_buf.st_dev > 8) & 0377,

stat_buf.st_dev & 0377);

printf("I-node: %d; Links: %d; Size: %ld\n", stat_buf.st_ino,

stat_buf.st_nlink, stat_buf.st_size);

if ((stat_buf.st_mode & S_ISUID) == S_ISUID)

printf("Set-user-ID\n");

if ((stat_buf.st_mode & S_ISGID) == S_ISGID)

printf("Set-group-ID\n");

if ((stat_buf.st_mode & S_ISVTX) == S_ISVTX)

printf("Sticky-bit set -- save swapped text after use\n");

printf("Permissions: %o\n", stat_buf.st_mode & 0777);

exit (0);

}

access()

To determine if a file is accessible to a program, the access() system call

may be used. Unlike any other system call that deals with permissions,

access() checks the real user-ID or group-ID, not the effective ones.

The prototype for the access() system call is:

int access(file_name, access_mode)

char *file_name;

int access_mode;

where file_name is the name of the file to which access permissions given in

access_mode are to be applied. Access modes are often defined as manifest

constants in /usr/include/sys/file.h. The available modes are:

Value Meaning file.h constant

----- ------ ------

00 existence F_OK

01 execute X_OK

02 write W_OK

04 read R_OK

These values may be ORed together to check for mone than one access

permission. The call to access() returns 0 if the program has the given

access permissions, otherwise -1 is returned and errno is set to the reason

for failure. This call is somewhat useful in that it makes checking for a

specific permission easy. However, it only answers the question "do I have

this permission?" It cannot answer the question "what permissions do I

have?"

The following example program demonstrates the use of the access() system

call to remove a file. Before removing the file, a check is made to make

sure that the file exits and that it is writable (it will not remove a

read-only file).

/* remove.c */

#include

#define ERR (-1)

int main();

int main(argc, argv)

int argc;

char *argv[];

{

if (argc != 2)

{

printf("Usage: %s filename\n", argv[0]);

exit (1);

}

if (access (argv[1], F_OK) == ERR) /* check that file exists */

{

perror(argv[1]);

exit (1);

}

if (access (argv[1], W_OK) == ERR) /* check for write permission */

{

fprintf(stderr,"File: %s is write protected!\n", argv[1]);

exit (1);

}

if (unlink (argv[1]) == ERR)

{

perror(argv[1]);

exit (1);

}

exit (0);

}

The vi Editor

2010-11-08T02:45:00.000-08:00

To write and edit some programs and scripts, we require editors. UNIX provides vi editor for BSD system – created by Bill Joy. Bram Moolenaar improved vi editor and called it as vim (vi improved) on Linux OS.

vi Basics

To add some text to a file, we invoke,

In all probability, the file doesn’t exist, and vi presents you a full screen with the filename shown at the bottom with the qualifier. The cursor is positioned at the top and all remaining lines of the screen show a ~. They are non-existent lines. The last line is reserved for commands that you can enter to act on text. This line is also used by the system to display messages. This is the command mode. This is the mode where you can pass commands to act on text, using most of the keys of the keyboard. This is the default mode of the editor where every key pressed is interpreted as a command to run on text. You will have to be in this mode to copy and delete text

For, text editing, vi uses 24 out of 25 lines that are normally available in the terminal. To enter text, you must switch to the input mode. First press the key i, and you are in this mode ready to input text. Subsequent key depressions will then show up on the screen as text input.

After text entry is complete, the cursor is positioned on the last character of the last line. This is known as current line and the character where the cursor is stationed is the current cursor position. This mode is used to handle files and perform substitution. After the command is run, you are back to the default command mode. If a word has been misspelled, use ctrl-w to erase the entire word.

Now press esc key to revert to command mode. Press it again and you will hear a beep. A beep in vi indicates that a key has been pressed unnecessarily. Actually, the text entered has not been saved on disk but exists in some temporary storage called a buffer. To save the entered text, you must switch to the execute mode (the last line mode). Invoke the execute mode from the command mode by entering a: which shows up in the last line.

The Repeat Factor

vi provides repeat factor in command and input mode commands. Command mode command k moves the cursor one line up. 10k moves cursor 10 lines up.

To undo whenever you make a mistake, press

Esc u

To clear the screen in command mode, press

ctrl-l

Don’t use (caps lock) - vi commands are case-sensitive

Avoid using the PC navigation keys

Input Mode – Entering and Replacing Text

It is possible to display the mode in which is user is in by typing,

:set showmode

Messages like INSERT MODE, REPLACE MODE, CHANGE MODE, etc will appear in the last line.

Pressing ‘i’ changes the mode from command to input mode. To append text to the right of the cursor position, we use a, text. I and A behave same as i and a, but at line extremes

I inserts text at the beginning of line. A appends text at end of line. o opens a new line below the current line

• r replacing a single character

• s replacing text with s

• R replacing text with R

• Press esc key to switch to command mode after you have keyed in text

Some of the input mode commands are:

COMMAND FUNCTION

i inserts text

a appends text

I inserts at beginning of line

A appends text at end of line

o opens line below

O opens line above

r replaces a single character

s replaces with a text

S replaces entire line

Saving Text and Quitting – The ex Mode

When you edit a file using vi, the original file is not distributed as such, but only a copy of it that is placed in a buffer. From time to time, you should save your work by writing the buffer contents to disk to keep the disk file current. When we talk of saving a file, we actually mean saving this buffer. You may also need to quit vi after or without saving the buffer. Some of the save and exit commands of the ex mode is:

Command Action

:W saves file and remains in editing mode

:x saves and quits editing mode

:wq saves and quits editing mode

:w save as

:w! save as, but overwrites existing file

:q quits editing mode

:q! quits editing mode by rejecting changes made

:sh escapes to UNIX shell

:recover recovers file from a crash

Navigation

A command mode command doesn’t show up on screen but simply performs a function.

To move the cursor in four directions,

k moves cursor up

j moves cursor down

h moves cursor left

l moves cursor right

Word Navigation

Moving by one character is not always enough. You will often need to move faster along a line. vi understands a word as a navigation unit which can be defined in two ways, depending on the key pressed. If your cursor is a number of words away from your desired position, you can use the word-navigation commands to go there directly. There are three basic commands:

b moves back to beginning of word

e moves forward to end of word

w moves forward to beginning word

Example,

5b takes the cursor 5 words back

3w takes the cursor 3 words forward

Moving to Line Extremes

Moving to the beginning or end of a line is a common requirement.

To move to the first character of a line

0 or |

30| moves cursor to column 30

$ moves to the end of the current line

The use of these commands along with b, e, and w is allowed

Scrolling

Faster movement can be achieved by scrolling text in the window using the control keys. The two commands for scrolling a page at a time are

ctrl-f scrolls forward

ctrl-b scrolls backward

10ctrl-f scroll 10 pages and navigate faster

ctrl-d scrolls half page forward

ctrl-u scrolls half page backward

The repeat factor can also be used here.

Absolute Movement

The editor displays the total number of lines in the last line

Ctrl-g to know the current line number

40G goes to line number 40

1G goes to line number 1

G goes to end of file

Editing Text

The editing facilitates in vi are very elaborate and invoke the use of operators. They use operators, such as,

d delete

y yank (copy)

Deleting Text

x deletes a single character

dd delete entire line

yy copy entire line

6dd deletes the current line and five lines below

Moving Text

Moving text (p) puts the text at the new location.

p and P place text on right and left only when you delete parts of lines. But the same keys get associated with “below” and “above” when you delete complete lines

Copying Text

Copying text (y and p) is achieved as,

yy copies current line

10yy copies current line & 9 lines below

Joining Lines

J to join the current line and the line following it

4J joins following 3 lines with current line

Undoing Last Editing Instructions

In command mode, to undo the last change made, we use u

To discard all changes made to the current line, we use U

vim (LINUX) lets you undo and redo multiple editing instructions. u behaves differently here; repeated use of this key progressively undoes your previous actions. You could even have the original file in front of you. Further 10u reverses your last 10 editing actions. The function of U remains the same.

You may overshoot the desired mark when you keep u pressed, in which case use ctrl-r to redo your undone actions. Further, undoing with 10u can be completely reversed with 10ctrl-r. The undoing limit is set by the execute mode command: set undolevels=n, where n is set to 1000 by default.

Repeating the Last Command

The . (dot) command is used for repeating the last instruction in both editing and command mode commands

For example:

2dd deletes 2 lines from current line and to repeat this operation, type. (dot)

Searching for a Pattern

/ search forward

? search backward

/printf

The search begins forward to position the cursor on the first instance of the word

?pattern

Searches backward for the most previous instance of the pattern

Repeating the Last Pattern Search

n repeats search in same direction of original search

n doesn’t necessarily repeat a search in the forward direction. The direction depends on the search command used. If you used? printf to search in the reverse direction in the first place, then n also follows the same direction. In that case, N will repeat the search in the forward direction, and not n.

Search and repeat commands

Command Function

/pat searches forward for pattern pat

?pat searches backward for pattern pat

n repeats search in same direction along which previous search was made

N repeats search in direction opposite to that along which previous search was made

Substitution – search and replace

We can perform search and replace in execute mode using :s. Its syntax is,

:address/source_pattern/target_pattern/flags

:1,$s/director/member/g can also use % instead of 1,$

:1,50s/unsigned//g deletes unsigned everywhere in lines 1 to 50

:3,10s/director/member/g substitute lines 3 through 10

:.s/director/member/g only the current line

:$s/director/member/g only the last line

Interactive substitution: sometimes you may like to selectively replace a string. In that case, add the c parameter as the flag at the end:

:1,$s/director/member/gc

Each line is selected in turn, followed by a sequence of carets in the next line, just below the pattern that requires substitution. The cursor is positioned at the end of this caret sequence, waiting for your response.

The ex mode is also used for substitution. Both search and replace operations also use regular expressions for matching multiple patterns.

The features of vi editor that have been highlighted so far are good enough for a beginner who should not proceed any further before mastering most of them. There are many more functions that make vi a very powerful editor. Can you copy three words or even the entire file using simple keystrokes? Can you copy or move multiple sections of text from one file to another in a single file switch? How do you compile your C and Java programs without leaving the editor? vi can do all this.

Source: Sumitabha Das, “UNIX – Concepts and Applications”, 4^th edition, Tata McGraw Hill, 2006

vi Editor Information

2010-11-04T05:31:00.001-07:00

To enter vi:

To enter vi from the command line, use any of the forms below:

vi file	Invoke vi editor on file
vi file1 file2	Invoke vi editor on files sequentially
view file	Invoke vi editor on file in read only mode
vi +n file	Invoke vi editor; editing starts on line n
vi -r file	Recover changes made to file after a system crash

ex Commands

To enter an ex command from vi:

To enter an ex command from vi, use this form:

:[address] command [options]
address is the line number or range of lines that are the object of command. If no address is given, the current line is the object of the command.
Address Symbols
In ex command syntax, address can be specified by any of the following:

1,$	All lines in the file.
x,y	Lines x through y.
x;y	Lines x through y, with current line reset to x.
0	Top of file.
.	Current line.
n	Absolute line number n.
$	Last line.
%	All lines; same as 1,$.
x-n	n lines before x.
x+n	n lines after x.
-[n]	One or n lines previous.
+[n]	One or n lines ahead.
'x	Line marked with x.
' '	Previous mark.
/pat / or ?pat ?	Ahead or back to line matching pat.

Option Symbols
In ex command syntax, options may be any of the following:

!	Indicates a variant form of the command, overriding the normal behavior.
count	The number of times the command is to be repeated. count cannot precede the command, because a number preceding an ex command is treated as a line address. d3 deletes three lines beginning with the current line; 3d deletes line 3.
file	The name of a file that is affected by the command. % stands for current file; # stands for previous file.

Commands

append

[address] a[!]
text
.
Append text at specified address, or at present address if none is specified. Add a ! to switch the autoindent setting that will be used during input. That is, if autoindent was enabled, ! disables it.

copy

[address] co destination
Copy the lines included in address to the specified destination address. The command t is a synonym for copy.

delete

[address] d [buffer]
Delete the lines included in address. If buffer is specified, save or append the text to the named buffer.

edit

e[!] [+n] [filename]
Begin editing on filename. If no filename is given, bring in another copy of the curent file. Add a ! to edit the new file even if the current file has not been saved since the last change. With the +n argument, begin editing on line n.

insert

[address] i[!]
text
.
Insert text at line before the specified address, or at present address if none is specified. Add a ! to switch the autoindent setting during input of text.

map

map char commands
Define a macro named char in visual mode with the specified sequence of commands. char is usually a single character, or the sequence #n, representing a function key on the keyboard.

mark

[address] ma char
Mark the specified line with char, a single lowercase letter. Return later to the line with 'x.

move

[address] m destination
Move the lines specified by address to the destination address.

next

n[!] [[+command]filelist]
Edit the next file from the command-line argument list. Use args to list these files. If filelist is provided, replace the current argument list with filelist and begin editing on the first file; if command is given (containing no spaces), execute command after editing the first such file.

put

[address] pu [char]
Restore previously deleted or yanked lines, from named buffer specified by char, to the line specified by address; if char is not specified, the last deleted or yanked text is restored.

quit

quit[!]
Terminate current editing session. Use ! to discard changes made since the lsat save. If the editing session includes additional files in the argument list that have not yet been accessed, quit by typing q! or by typing q twice.

read

[address] r filename
Copy the text of filename at the specified address. If filename is not specified, the current filename is used.

read

[address] r !command
Read in the output of command into the text after the line specified by address.

substitute

[address] s [/pattern/repl/] [options]
Replace each instance of pattern on the specified lines with repl. If pattern and repl are omitted, repeat last substitution. An option of g substitutes all instances of pattern on the line. An option of c prompts for confirmation before change.

undo

u
Reverse the changes made by the last editing command.

write

[address] w[!] [[>>]filename]
Write lines specified by address to filename, or full contents of buffer if address is not specified. If filename is also omitted, save the contents of the buffer to the current filename. if >> filename is used, write contents to the end of the specified filename. Add a ! to force the editor to write over any current contents of filename.

yank

[address] ya [char] [count]
Place lines specified by address in named buffer indicated by char, or if no char is specified place in general buffer.

[address] z [type] [count]
Print a window of text with line specified by address at the top. type can be one of:

+	Place specified line at the top of the window (default).
-	Place specified line at bottom of the window.
.	Place specified line in the center of the window.
^	Print the previous window.
=	Place specified line in the center of the window and leave the current line at this line.

count specifies the number of lines to be displayed.

[address] !command
Execute command in a shell. If address is specified, apply the lines contained in address as standard input to command, and replace the lines with the output.

vi Commands

vi has two "modes", command and insert. Insert mode (text entry) is terminated by , which toggles back to command mode. in command mode is ignored.

In vi, commands have the following general form:
[n] operator [m] object
The basic editing operators are:

c	Begin a change.
d	Begin a deletion.
y	Begin a yank (or copy).

If the current line is the object of the operation, then the operator is the same as the object: cc, dd, yy. Otherwise, the editing operators act on objects specified by cursor-movement commands or pattern-matching commands. n and m are the number of times the operation is performed, or the number of objects the operation is performed on. If both n and m are specified, the effect is n x m.
An object can represent any of the following text blocks:

word	Includes characters up to a space or punctuation mark. A capitalized object is a variant form that recognizes only blank spaces.
sentence	Is up to ., !, ? followed by two spaces.
paragraph	Is up to next blank line or paragraph macro defined by para=option.
section	Is up to next section heading defined by sect=option.

Examples

2cw	Change the next two words.
d}	Delete up to next paragraph.
d^	Delete back to beginning of line.
5yy	Copy the next five lines.
y]]	Copy up to the next section.

Movement Commands

Character:
h,j,k,l	Left, down, up, right (direction arrows)
Text:
w, W, b, B	Forward, backward by word
e, E	End of word
), (	Beginning of next, previous sentence
},{	Beginning of next, previous paragraph
]],[[	Beginning of next, previous section
Lines:
0, $	First, last position of current line
^	First character of current line (ignore spaces)
+, -	First character of next, previous line
n\|	Column n of current line
H	Top line of screen
M	Middle line of screen
L	Last line of screen
nH	n (number) of lines after top line
nL	n (number) of lines before last line
Screens:
Ctrl-F	Scroll forward one screen
Ctrl-B	Scroll backward one screen
Ctrl-D, Ctrl-U	Scroll down, up one-half screen
Ctrl-E, Ctrl-Y	Show one more line at bottom, top of window
z	Reposition line with cursor: to top of screen
z.	Reposition line with cursor: to middle of screen
z-	Reposition line with cursor: to bottom of screen
Ctrl-L, Ctrl-R	Redraw screen (without scrolling)
Searches:
/pattern	Search forward for pattern
?pattern	Search backward for pattern
n, N	Repeat last search in same, opposite direction
/, ?	Repeat previous search forward, backward
fx	Search forward for character x in current line
Fx	Search backward for character x in current line
tx	Search forward for character before x in current line
Tx	Search backward for character after x in current line
;	Repeat previous current-line search
,	Repeat previous current-line search in opposite direction
Line number:
Ctrl-G	Display current line number and file name
nG	Move to line number n
G	Move to last line in file
:n	Move to line number n
Marking position:
mx	Mark current position as x
`x	Move cursor to x (grave character)
``	Return to previous mark or context (two grave characters)
'x	Move to beginning of line containing mark x (single quote)
''	Return to beginning of line containing previous mark (two single quotes)

Editing Commands

Insert:
i, a	Insert text before, after cursor
I, A	Insert text at beginning, end of line
o, O	Open new line for text below, above cursor
Change:
r	Change character (no required)
cw	Change word
cc	Change currnt line
C	Change to end of line
R	Type of characters
s	Delete character and substitute text
S	Delete current line and substitute text
Delete, move:
x	Delete character
X	Delete character before cursor
dw	Delete word
dd	Delete current line
D	Delete to end of line
p, P	Put deleted text after, before cursor
"np	Put text from delete buffer n after cursor (for last nine deletions)
Yank:
yw	Yank (copy) word
yy	Yank current line
"ayy	Yank current line into named buffer a
p, P	Put yanked text after, before cursor
"aP	Put text from buffer a before cursor
Other commands:
.	Repeat last edit command
u, U	Undo last edit; restore current line
J	Join two lines

Exit Commands

ZZ	Write (save) and quit file
:x	Write (save) and quit file
:wq	Write (save) and quit file
:w	Write (save) file
:w!	Write (save) file (overriding protection)
:30, 60w newfile	Write from line 30 through 60 as newfile
:30, 60w >> file	Write from line 30 through line 60 and append to file
:w %.new	Write current buffer named file as file.new
:q	Quit file
:q!	Quit file (overriding protection)
Q	Quit vi and invoke ex
:e file2	Edit file2 without leaving vi
:n	Edit next file
:e!	Return to version of current file at time of last write (save)
:e#	Edit alternate file
%	Current filename
#	Alternate filename

UNIX SHELL COMMANDS

2010-11-03T22:17:00.000-07:00

What is a Shell?

A shell is basically the user interface in UNIX. It is a program that allows the system to understand your commands. The shell has three main uses:

· Interactive Use

When used interactively, a shell waits for you to type a command at the prompt

· Customizing your UNIX session

The shell lets you define environment variables that control the behavior of your UNIX session, such as setting the appearance of the screen.

· Programming

The UNIX shell provides a number of commands that can be used to create programs called shell scripts. Scripts are useful for executing a series of individual commands. They can also execute commands in loops (repeatedly) or conditionally (if-else). There are three main types of shells: the Bourne shell, the Korn shell, and the C shell. Most people at Brown use the tcsh, which is an enhanced C shell.

xterm

The command xterm starts up a new shell. If you are using this command from a shell, however, you will want to put it in the background using the '&' operator. Otherwise you will be running the shell from the shell in which it was executed from and won't be able to use that shell anymore. A useful option for the xterm is -e which allows you to specify the program to be run in the xterm window. If I want a login shell, for example, I type xterm -e /bin/login & . This provides a new shell with a login prompt which allows someone to login to the computer.

setenv

In UNIX, the system has to have a way to store certain settings that the shell and various programs use. This information is stored in "environment variables" each of which has a name (like DISPLAY, PATH, TIMEZONE, HOME) and a value. There is no standard UNIX command to set these environment variables, but the C-Shell (which you are probably using a form of) has a built-in command called setenv which you can use to set one of these variables.

printenv

The C-Shell command (again, not a real UNIX command) printenv will show you the contents of an environment variable.

Some Useful Operators for the csh or tcsh:

*	match any characters in a filename (wild-card)
\|	take output from one command and use it as input for a second command (pipe)
>	redirect output (over-writes)
>>	append output
&	run process in the background

Simple Commands

man

This command displays information from online documentation. It will give you a manual page for virtually any command (or programming function) available on the system, which describes what the command does and how it is used. Each command in this document is linked to its man page for easy reference.

finger

Locate any user using the finger command. This command provides useful user information such as login name, the person's real name, what computers the user is currently logged in on, when the users last login was, and information about when the user last received and checked e-mail. It also includes the information listed in the files .plan and.project. These files will be explained later, but here is an example of what you would see if you typed in a shell finger dmb. Alternatively, you could use finger with a person's first or last name, and it will find all the login names for that name. If fingering a username, you can abbreviate this command to f [username]. You can also finger at a machine and get back who is logged into that machine and from where. Typing finger @cslab0a will return all the users logged on to cslab0a. Typing finger by itself will tell you who is logged on to the machine you are on.

Files and Directories

The cd command lets you navigate the filesystem. The UNIX filesystem is organized as a heirarchy where each level is separated by a "/". The "root" directory is simply called /. So, if I am in / I can type cd u/ck and I will be in /u/ck, ck's home directory, since it will change directories relative to my current path. If I codefix the directory name with a /, it will jump to an absolute location so typing cd /u/ljc from /u/ck will put you into /u/ljc. Also, the directory ".." is one level up in the heirarchy, so from /u/adt typing cd .. will move you up one directory into /u.

This command lists all directories and files contained inside the current directory, when typed alone. The ls commands has a number of options that let you display a variety of information about what is contained in a directory. Some useful options include:

-a

Lists all the files in a directory, including the normally hidden . files.

-F

Flag filenames by adding / to directories, * to executable files, and @ to symbolic links.

-l

The long format listing includes permissions, owner, size and modification time.

-R

lists subdirectories as well as directories recursively.

So, for example, if i want to flag filenames when using the ls command, I can type ls -F.

These commands allow you to display the contents of a file, which is useful if you want to scroll through a file that is too long to fit within one screen. Two simple ways to do this are to use the more and less commands. These commands are similar in that you can read, but not edit a file using this command.

Using more, press the return key to view the next line of text, the spacebar to read the next page of text, q to quit, and / to search.

You can use the same commands in less, but in addition you can use the arrow keys to navigate by line, and u to move up a page. A useful option is -N, which numbers the line in the files.

To use more or less, type more or less and the name of the file you want to read. Type any option you want to use before the filename. As with any command in this guide, for more options or more information about a command, see the man pages.

This allows you to change the name of a file, or move files or directories around. To create a new file in UNIX we simply try to open a file that does not exist in a text editor, described below. UNIX will then create a new file with this name. If we want to change the name of a file or move it to a directory other than the one it was created in, we use the mv command. To use this command, we type mv then the name or names of the files we want to move and then the directory or new file name we want to move to. For example, if we have a file named foo and we want to rename it foo.html, we type mv foo foo.html. Now if we want to move foo.html to dmb's home directory, we type mv foo.html /u/dmb.

To copy a file into a file with a different name, or into another directory, we use the cp command. Above, we moved foo to foo.html, but if we want to copy foo into a file called foo.html, but don't want to remove foo, we type cp foo foo.html. Likewise, we can copy a file into another directory: cp foo.html /u/dmb . If we want to copy a file from a different directory into a current directory, we can use a '.' to mean the current directory. So to copy foo.html from dmb's home directory into the current directory, type cp /u/dmb/foo.html .

To remove a file we no longer need, we use the rm command. This command has a number of useful options:

-i

This makes rm interactive, and asks the user if it wants to rm the file or files named. The user responds with y or n.

-r

In UNIX, an * can be used as a wild card. Typing rm *.html removes all files whose name ends in ".html". Likewise, typing rm * removes all the contents of a directory. However, to recursively remove the contents of all subdirectories, you must use the -r option. The command can also be used to remove a not-empty directory and its contents by specifying a directory name. Be warned! This is a dangerous command to use, because if the directory contains files, you may be unknowingly removing them when you remove the entire directory.

mkdir

To create a new directory, use the mkdir command. This will create a new directory with the specified name in the current directory. So to create the directory JavaWork in the current directory, we type mkdir JavaWork.

rmdir

To remove an empty directory, or to remove the name of a directory, but not its contents type rmdir and the directory name. For a directory that is not empty, the rm -rcommand can be used.

pwd

This commands prints out the full pathname of the current directory. So if I type pwd while I am in ljc's home directory, UNIX responds /home/ljc .

chmod

The chmod command is a little more complicated than some of the other commands, but extremely important. Everything you create has permissions for the people who can read, write to, or execute the file or directory you create. The command ls -l lists the permissions of a file. If you want to change who can read a file you use the command chmod. You must then specify who you are changing permission for and what permission you are changing it to. Who and the permission are connected by either a '+' (to add a permission), a '-' (to remove a permission), or an '=' (to assign a permission and remove other permissions). Who can be either 'u' for the user, 'g' for the group the user belongs to (all users belong to at least one group), 'o' for everyone else, and 'a' all. The common permissions include r (read), w (write), and x (execute - for directories and executable files). Here are some examples of how to change permissions on the file foo.html:

Add read permission for all:

chmod a+r foo.html

Add read and write permissions for the group:

chmod g+rw foo.html

remove execute permission for others:

chmod o-x foo.html

chgrp

Often users belong to more than one group. To change the group that has permission on a certain file, use the chgrp command. To use this command, type chgrp, the name of the new group you want to assign the file to, and the file name. For example, to assign foo.html to the cs015 group, type chgrp cs015 foo.html.

grep

The grep command allows you to search one or more files for lines that match an expression. Two useful options of this command are:

-i

Ignores uppercase and lowercase distinctions

-n

Prints lines and their line numbers.

To use the command, type grep, any options you want, the expression to search for and the files to search in. For example to search for the word UNIX in foo.html, typegrep UNIX foo.html .

cat

The command cat simply takes a file or multiple files and outputs them. While it may seem simple, it allows you to perform many powerful operations. UNIX has notions of standard input and standard output. Your screen is typically standard output, so when you run a command, the output appears on your screen. However, you can redirect standard output, so it goes to a file. The > operator will redirect standard output to a file, and the >> operator will append standard output to a file. So, cat a b > foo.html will create a file, foo.html that consists of the contents of a followed by the contents of b. If we then type cat c >> foo.html foo.html will contain the contents of a then b then c. Another redirection operator, called pipe (the '|' key), takes the output of one operation and uses it as the input to another operation. So cat foo.html | mail dmb will take the output from foo. html and use it as the input for and e-mail to dmb.

This command prints the disk usage (without the -k option, it prints out the number of 512-byte blocks used). The -k option writes the files sizes in units of 1024 bytes, rather than 512. If you name a directory, it will print out the size of that directory and all sub-directories, or the default is the home directory.

gzip/gunzip

The gzip command reduces the size of the named files by compressing it. To read a gzipped file, you must first uncompress it by using the gunzip command.

compress/uncompress

Basically the same as gzip, compress reduces the size of the named files. To read the files, use uncompress.

touch

The touch command updates the access and modification times of the named files to the current time and date. The command is useful because some commands rely on a file's access and modification times.

tail

The head command prints only the first ten lines of the named file and the tail command prints only the last ten lines. If you want head or tail to print out a different number of lines, you can specify the length as - . For example, head -20 will print out the first 20 lines of a file.

sort

To sort the lines of a file, typically in alphabetical or numerical order, use the sort command. One useful option is -r, sort in reverse order.

The ln command creates links for files. For example, if you type ln foo.html foo , foo becomes a psuedonym for foo.html. If foo is an already existing file, it is overwritten. If foo is a directory, however, a link named foo.html is created in foo. The -s option creates a symbolic link, which lets you link across file systems and also lets you see the name of the link when you run ls -l.

Communication Commands

zwrite

The command zwrite sends a message to another user by using the zephyr notification service, by displaying a window with a message on the recipient's screen. Zephyr is not available on all UNIX systems, but zwrite is a commonly used command at Brown.

zwgc

Zephyr Windgowgram Client program, or zwgc, is the main zephyr client used for zwrites. To receive zwrites, a user must be running zwgc. This can be done by using the command zwgc, or if you want to receive zwrites in a shell, type zwgc -ttymode .

write

The write command is used when zwrite is not available. It allows you to write to the shell (usually the console) of another user. In order to write to a user, however, you must first remotely log on (rsh) to their machine. To end a message, type control-d on a line by itself.

talk

The command talk creates a two-way, screen-oriented communication program. It allows users to type simultaneously, with their output displayed in separate regions of the screen. To send a talk request, type talk and then @. So, for example, to send a talk request to dmb, who is logged on to cslab0a, type talk dmb@cslab0a . The recipient of the request is then prompted for a response.

mesg

The mesg command changes the ability of others to send write or zwrite messages or talk requests to a user. Typing mesg n or mesg -n forbids messages, while mesg yor mesg -y allows messages.

mail

The mail command is a quick and easy way to send an email to someone. Just type mail and the address of the person you want to mail. You will then be prompted for a subject and any cc's. Then just type your message and control-d on a line by itself to send the mail.

pine

There are a number of ways to read e-mail in UNIX. One option is pine. It a screen-oriented tool with limited functions for message-handling. You can also use pine to send a quick email without opening your mail directory and inbox (which pine does when you type the pine alone) by typing pine followed by an e-mail address.

tin

There are also a number of options to use to read newsgroups. One popular news reader is tin, which like pine is screen-oriented with functions to read and compose news. When you start up tin, it will show a list of the newsgroups found in you .newsrc file in your home directory. You can also specify a newsgroup for tin to open by typing tin and then the name of the newsgroup.

Process Commands

Everything you run from your account is called a process. So when you are running a shell, netscape and emacs, you are running three processes. There are a number of useful commands for dealing with processes.

Without specifying options, ps lists all the active processes running from a given shell. A more useful way of using the ps command is to use the -u option, which you specify a user, and this lists all the processes that user is running. Each process listed has a 'PID' (process id) number associated with it. This is an important number!

kill

Use kill with a PID number to terminate a process. Use the ps command (above) to find out the PID for a process. Then type kill to stop the process.

nice

All commands that you run have a priority. To give a command a lower priority, use nice with the command name. You can specify how "nice" you want to be by using the -noption with a number from 1 to 19. A higher number means a lower priority. This command is often used if you are running something on a machine and want to give others priority when they are running processes that are more important.

top

The command top displays the top 15 processes on the system and periodically updates this information. If you specify a number, then that number processes will be displayed instead of the default.

If a process is running in the background, fg runs it in the foreground.

If a process is running in the foreground, bg runs it in the background.

Text Editors

There are a number of text editors available when using the UNIX operating system. Some work from with in the shell that you start it in. Start up these editors by using their name and then the name of the file you wish to edit. A new file is indicated by a name that does not already exist in the directory you are in. Other editors start up in their own windows when you start them from a shell. Read the man pages for more information about how to use the different text editors.

Text Editors in a Shell:

emacs
pico
vi

Text Editor Applications:

xemacs
emacs-19

Using Printers

In UNIX, you can print a file without opening it in a text editor using various commands. When you print to one of Brown's printers, your file will be printed with a "banner" sheet that identifies who the print job is for. The CS department has various locations to print to. The printer for undergraduates is cis. The department also has three printers for staff, namely ps1, ps2 and ps3. The 'ps' is short for POSTSCRIPT, which is what our printers use to understand and print the files.

lpr

The command lpr prints the named files to either the user's default printer (for most undergraduates, cis) or to the named printer using the -P option. For example, to print foo.html to cis (and cis is not the default printer), type lpr -Pcis foo.html .

lpq

To print out the printer queue to check the status of a print job, use the lpq command. To specify a printer, use the -P option as used above. So, if I want to print out the queue for cis, I type lpq -Pcis .

lprm

The command lprm allows you to remove a job from the printer queue. To use the command, you can specify the job number by using the lpq command to determine what the number is. So if I find out from lpq that the job I want to remove is 787, I can type lprm 787 to remove the job from the printer queue. Using the command alone removes the first of your jobs in the printer queue.

The pr command is most commonly used at brown with the -f option to print multiply files without banner sheets between the files. For example, if I want to print foo1, foo2 and foo3, I type pr -f foo1 foo2 foo3 | lpr . The pipe (|) operation takes the output from the first command and uses it as the input for the second command. The pr command is basically used to apply options to, and then sent to the printer by piping the output to the lpr command. For more options, see the man pages.

enscript

The printer at Brown reads mainly POSTSCRIPT files. The enscript command converts text files to POSTSCRIPT format for printing.

Logging On to Machines

Logging on to a machine on console (at the terminal) is simple. But what if you want to use someone else's machine to log into your account, or log into another machine remotely. This is useful when you are running a number of processes on your machine and it would be faster to run some on another machine.

The su command allows you to become the "super-user" or another user in a shell. In order to use su, the password of the super-user must be supplied when prompted. If the first argument to su is a dash (-), for example su - dmb , the computer will go through the entire login sequence, where as without the dash, the environment will remain unchanged.

rsh

The rsh command creates a remote shell by connecting to the specified computer. If you specify a command, the command will be executed on the specified computer. A useful option is -l, which allows you to log in as a specified user. So, typing rsh cslab0a -l dmb will allow dmb to remotely log on to cslab0a from the computer she is on. She will have to type her password at the prompt for this to be successful, however. If dmb is already logged on the computer she is trying to rsh another machine from, she does not have to use the -l option. She can simply type rsh cslab0a .

rlogin

The command rlogin creates a remote login session from your terminal to the machine specified. Once again, you can specify a user name using the -l option. For dmb to remotely log on to another machine from someone else's account she would type rlogin cslab0a -l dmb and type her password at the prompt. To remotely log on from her own account to another machine she would simply need to type rlogin cslab0a .

X server and DISPLAY

When you are logged onto console on a machine, you own the X server, or what can be displayed on your screen. In order for other users or other machines to use your DISPLAY, you must give them permission, and in order for a user to use the DISPLAY of another machine, he or she must set the DISPLAY to be that computer.

xhost

The xhost command is used to add and delete computers or users to the list allowed to make connections to the x server, which enables something to appear on the screen. Use xhost + to add permission, or xhost - to remove a permission. Using xhost + without a name allows anyone to use your xserver. The commandxhost + localhost can be used to allow any user logged on to your machine to use the display.

setenv DISPLAY

The command setenv is used to set the various environment variables. A common variable that is often changed is the 'DISPLAY' variable, which determines where something will be displayed. You can set DISPLAY to any computer that has given your computer, or a user, permission to use the x server. So, for example, if you are a super-user on a computer and you want to set DISPLAY to localhost, you would type setenv DISPLAY localhost:0

Miscellaneous Commands

yppasswd

Everyone has a password to get into their account for security purposes. You can change your network password using the yppasswd command. After typing the command, it will prompt you for your old password (for security reasons) and then ask you the new password twice to prevent mistakes.

cal

The cal command prints a calendar to standard output. You can specify a month (using the number for the month) and a year, or just a year to print a calendar for an entire year. If you do not specify anything, a calendar for the current month is printed. So to print the calendar for December 1999, type cal 12 1999 .

date

The date command simply returns the current date and time.

whoami

The whoami command displays the login name of the current effective user. If you have used su to adopt another user, whoami wil report the login name associated with that user ID. For a command similar to whoami, but with more options, see the man pages for who.

which

The command which takes a list of specified names and looks for what would be executed had these names been given as commands. UNIX allows you to alias commands to a shorter name in your .alias file. If you have aliased a command, the which expands the name. For example, if I have the command 'zwrite' aliased to 'z' in my .alias file, and I type which z , the command returns z: aliased to zwrite .

loc

The command loc returns the address of a user. This command is extremely useful when used with the -g option, which locates all users in the specified group. For example, if i wanted to locate all of the cs015 tas, I would type loc -g cs015ta .

groups

The groups command will list all of the groups a specified user belongs to. If a user is not specified, the groups for the current user will be listed.

grplist

The grplist command will list all the users in a specified group.

telnet

To create a connection to a remote system, use telnet.

keyinit

In order to telnet into the Brown Computer Science department's telnet host, you must use the S/Key authentication system. The keyinit command is used to initialize the system or change your password so you can use S/Key one-time passwords to login. This lets you avoid sending a real password over the network.

lw, sunfree, sunlist

These are commands used for the sun lab at Brown University. They provide information in various formats about who is logged in on console on what computer in the sunlab. Sorry, there are no man pages for these commands.

xstartfm

Framemaker is a word processing and desktop publishing package. It is often used to make lecture slides in the Brown cs department. To start framemaker, use the xstartfm command. It is often helpful to run framemaker in the background by using the & operator.

xcolorsel

xcolorsel displays the colors available (usually the contents of the rgb.txt file). Each color will consist of three intergers (usually 0-255) representing the RGB values and a nickname for the color. You may want to run this command in the background using the & operator.