The Mad Hatter

mad as a hatter

Resize Linux storage with LVM

ou need more space on your server/desktop/etc.. This guide will help you add it to the volume you’re currently using on your system. For the sake of making this an easy-to-follow guide, we’ll assume that you already have the new drive showing in your Linux installation from fdisk, but it’s not yet usable (ie: you made it available from shared storage, you installed a new harddrive, etc..).

First, let’s see if the system can see the new storage:

# fdisk -l
Disk /dev/sdc doesn't contain a valid partition table

Note: you’ll also see information about your current drives..

First things first, create a partition on the drive using fdisk, cfdisk, etc.. toggle it type ’8e’.

# fdisk /dev/sdc

Now it should show up a little better in the fdisk -l command.

Next, create the physical volume on the new partition on the new drive:

# pvcreate /dev/sdc1
Physical volume "/dev/sdc1" successfully created

Now, find out the volume group and logical volume names on your system. Make note of these..

# vgdisplay
# lvdisplay

We’ll assume that the information is:
Volume Group: test_vg
Logical Volume: test_lv

Now, use vgextend to add your new drive to the existing volume group:

# vgextend test_vg /dev/sdc1
  Volume group "test_vg" successfully extended

You can see, by viewing ‘vgdisplay’ again, it will show you the added space:

# vgdisplay

Now, pull it across into the logical volume:

# lvextend -l +100%FREE /dev/test_vg/test_lv
  Extending logical volume test_lv to 6.88 TiB
  Logical volume test_lv successfully resized

You’re almost home free – now, vgdisplay shows the correct info and so does lvdisplay – but a simple df -h still shows the filesystem size has not changed. Let’s tell the filesystem to stretch out across the full logical volume:

# resize2fs /dev/test_vg/test_lv
resize2fs 1.41.12 (17-May-2010)
Filesystem at /dev/test_vg/test_lv is mounted on /TEST; on-line resizing required
old desc_blocks = 347, new_desc_blocks = 441
Performing an on-line resize of /dev/test_vg/test_lv to 1848109056 (4k) blocks.
The filesystem on /dev/test_vg/test_lv is now 1848109056 blocks long.

And to see how it worked out:

# df -h
Filesystem                    Size  Used Avail Use% Mounted on
/dev/mapper/test_vg-test_lv   6.8T  4.9T  1.9T  73% /TEST

It worked!

Linux Tips

Here is a selection of command-line tips that I’ve found useful when working on Linux. The emphasis is on somewhat less-known techniques that are generally important or useful to technical users. It’s a bit long, and users certainly don’t need to know all of them, but I’ve done my best to review that each item is worth reading in terms of projected time savings, if you use Linux heavily.

To get more information on a command mentioned, first try “man <command name>”. In some cases, you must install a package for this to work — try aptitude or yum. If that fails, Google it.
Basics

Learn basic bash. Actually, read the whole bash man page; it’s pretty easy to follow and not that long. Alternate shells can be nice, but bash is powerful and always available (learning mainly zsh or tcsh restricts you in many situations).
Learn vim. There’s really no competition for random Linux editing (even if you use Emacs or Eclipse most of the time).
Know ssh, and the basics of passwordless authentication, via ssh-agent, ssh-add, etc.
Be familiar with bash job management: &, Ctrl-Z, Ctrl-C, jobs, fg, bg, kill, etc.
Basic file management: ls and ls -l (in particular, learn what every column in “ls -l” means), less, head, tail and tail -f, ln and ln -s (learn the differences and advantages of hard versus soft links), chown, chmod, du (for a quick summary of disk usage: du -sk *), df, mount.
Basic network management: ip or ifconfig, dig.
Know regular expressions well, and the various flags to grep/egrep. The -o, -A, and -B options are worth knowing.
Learn to use aptitude or yum (depending on distro) to find and install packages.

Everyday use

In bash, use Ctrl-R to search through command history.
In bash, use Ctrl-W to kill the last word, and Ctrl-U to kill the line. See man readline for default keybindings in bash. There are a lot. For example Alt-. cycles through prevous arguments, and Alt-* expands a glob.
To go back to the previous working directory: cd -
Use xargs (or parallel). It’s very powerful. Note you can control how many items execute per line (-L) as well as parallelism (-P). If you’re not sure if it’ll do the right thing, use xargs echo first. Also, -I{} is handy. Examples:

find . -name \*.py | xargs grep some_function

cat hosts | xargs -I{} ssh root@{} hostname

pstree -p is a helpful display of the process tree.
Use pgrep and pkill to find or signal processes by name (-f is helpful).
Know the various signals you can send processes. For example, to suspend a process, use kill -STOP [pid]. For the full list, see man 7 signal
Use nohup or disown if you want a background process to keep running forever.
Check what processes are listening via netstat -lntp. See also lsof.
In bash scripts, use set -x for debugging output. Use set -e to abort on errors. Consider using set -o pipefail as well, to be strict about errors (though this topic is a bit subtle). For more involved scripts, also use trap.
In bash scripts, subshells (written with parentheses) are convenient ways to group commands. A common example is to temporarily move to a different working directory, e.g.

# do something in current dir

(cd /some/other/dir; other-command)

# continue in original dir

In bash, note there are lots of kinds of variable expansion. Checking a variable exists: ${name:?error message}. For example, if a bash script requires a single argument, just write input_file=${1:?usage: $0 input_file}. Arithmetic expansion: i=$(( (i + 1) % 5 )). Sequences: {1..10}. Trimming of strings: ${var%suffix} and ${var#prefix}. For example if var=foo.pdf, then echo ${var%.pdf}.txt prints “foo.txt”.
The output of a command can be treated like a file via <(some command). For example, compare local /etc/hosts with a remote one: diff /etc/hosts <(ssh somehost cat /etc/hosts)
Know about “here documents” in bash, as in cat <<EOF ….
In bash, redirect both standard output and standard error via: some-command >logfile 2>&1. Often, to ensure a command does not leave an open file handle to standard input, tying it to the terminal you are in, it is also good practice to add “</dev/null”.
Use man ascii for a good ASCII table, with hex and decimal values.
On remote ssh sessions, use screen or dtach to save your session, in case it is interrupted.
For web debugging, curl and curl -I are handy, and/or their wget equivalents.
To convert HTML to text: lynx -dump -stdin
If you must handle XML, xmlstarlet is good.
For Amazon S3, s3cmd is convenient (albeit immature, with occasional misfeatures).
In ssh, knowing how to port tunnel with -L or -D (and occasionally -R) is useful, e.g. to access web sites from a remote server.
It can be useful to make a few optimizations to your ssh configuration; for example, this .ssh/config contains settings to avoid dropped connections in certain network environments, not require confirmation connecting to new hosts, forward authentication, and use compression (which is helpful with scp over low-bandwidth connections):

TCPKeepAlive=yes

ServerAliveInterval=15

ServerAliveCountMax=6

StrictHostKeyChecking=no

Compression=yes

ForwardAgent=yes

If you are halfway through typing a command but change your mind, hit Alt-# to add a # at the beginning and enter it as a comment (or use Ctrl-A, #, enter). You can then return to it later via command history.

Data processing

Know about sort and uniq (including uniq’s -u and -d options).
Know about cut, paste, and join to manipulate text files. Many people use cut but forget about join.
It is remarkably helpful sometimes that you can do set intersection, union, and difference of text files via sort/uniq. Suppose a and b are text files that are already uniqued. This is fast, and works on files of arbitrary size, up to many gigabytes. (Sort is not limited by memory, though you may need to use the -T option if /tmp is on a small root partition.)

cat a b | sort | uniq > c # c is a union b

cat a b | sort | uniq -d > c # c is a intersect b

cat a b b | sort | uniq -u > c # c is set difference a – b

Know that locale affects a lot of command line tools, including sorting order and performance. Most Linux installations will set LANG or other locale variables to a local setting like US English. This can make sort or other commands run many times slower. (Note that even if you use UTF-8 text, you can safely sort by ASCII order for many purposes.) To disable slow i18n routines and use traditional byte-based sort order, use export LC_ALL=C (in fact, consider putting this in your .bashrc).
Know basic awk for simple data munging. For example, summing all numbers in the third column of a text file: awk ‘{ x += $3 } END { print x }’. This is probably 3X faster and 3X shorter than equivalent Python.
Use shuf to shuffle or select random lines from a file.
Know sort’s options. Know how keys work (-t and -k). In particular, watch out that you need to write -k1,1 to sort by only the first field; -k1 means sort according to the whole line.
Stable sort (sort -s) can be useful. For example, to sort first by field 2, then secondarily by field 1, you can use sort -k1,1 | sort -s -k2,2
If you ever need to write a tab literal in a command line in bash (e.g. for the -t argument to sort), press Ctrl-V <tab>.
For binary files, use hd for simple hex dumps and bvi for binary editing.
Also for binary files, strings (plus grep, etc.) lets you find bits of text.
To convert text encodings, try iconv. Or uconv for more advanced use (it supports some advanced Unicode things, such as transforms for normalization, accent removal, etc.).
To split files into pieces, see split (to split by size) and csplit (to split by a pattern).

System debugging

To know disk/cpu/network status, use iostat, netstat, top (or the better htop), and (especially) dstat. Good for getting a quick idea of what’s happening on a system.
To know memory status, run and understand the output of free and vmstat. In particular, be aware the “cached” value is memory held by the Linux kernel as file cache, so effectively counts toward the “free” value.
Java system debugging is different kettle of fish, but a simple trick on Sun’s and some other JVMs is that you can run kill -3 <pid> and a full stack trace and heap summary (including generational garbage collection details, which can be highly informative) will be dumped to stderr/logs.
Use mtr as a better traceroute, to identify network issues.
To find which socket or process is using bandwidth, try iftop or nethogs.
The ab tool (comes with Apache) is helpful for quick-and-dirty checking of web server performance.
For more serious network debugging, wireshark or tshark.
Know strace, and that you can strace a running process (with -p). This can be helpful if a program is failing, hanging, or crashing, and you don’t know why.
Know about ldd to check shared libraries etc.
Know how to connect to a running process with gdb and get its stack traces.
Use /proc. It’s amazingly helpful sometimes when debugging live problems. Examples: /proc/cpuinfo, /proc/xxx/cwd, /proc/xxx/exe, /proc/xxx/fd/, /proc/xxx/smaps.
When debugging why something went wrong in the past, sar can be very helpful. It shows historic statistics on CPU, memory, network, etc.
Use dmesg whenever something’s acting really funny (it could be hardware or driver issues).

Most commands in linux stand on their own and do not require cat. For example the examples above using cat, sort, and uniq could have been greatly simplified and used less overhead (especially if larger files are used).

Example:

[user@host ~ ]$ echo 1234 > a
[user@host ~ ]$ echo 5678 > b
[user@host ~ ]$ sort a b
1234
5678
[user@host ~ ]$ cat a b |sort
1234
5678

As you can see above the command yielded the same results without cat! The cat often misused, but for now we will abuse it since I want to make this easy for people.

You can transfer files using ssh (ad not just by scp) sshfs.

Mounting a filesystem (requires fuse and fuse-ssh):

[user@host ~ ]$ mkdir ./remote_fs
[user@host ~ ]$ sshfs example.com:/stuff ./remote_fs

Transfering files:
[user@host ~ ]$ tar zcvf – ./my-dir | ssh remotehost.com “cat > /backup/mydir.tar.gz”

[user@host ~ ]$ tar cvf – ./my-dir | ssh remotehost.com tar xvf -

You can even use flags to make rsync use ssh so that the transfer is encrypted.

[user@host ~ ]$ rsync -azve ssh ./my-dir remotehost.com:/home/use/

Once go to man pages for more details, I can not repeat this enough… RTFM (Read The F*@$%g Manual).

tcpdump tips

Packet capture is one of the most fundamental and powerful ways to do network analysis. You can learn virtually anything about what is going on within a network by intercepting and examining the raw data that crosses it. Modern network analysis tools are able to capture, interpret and describe this network traffic in a human-friendly manner.

tcpdump is one of the original packet capture (or “sniffing”) tools that provide these analysis capabilities, and even though it now shares the field with many other utilities, it remains one of the most powerful and flexible.

If you think that tcpdump has been made obsolete by GUI tools like Wireshark, think again. Wireshark is a great application; it’s just not the right tool for the job in every situation. As a refined, universal, utility tcpdump satisfies a different type of need.

One of tcpdump’s greatest strengths is its convenience. It uses a “one-off-command” approach that lends itself to quick, on-the-spot answers. It works through an SSH session, doesn’t need X and is more likely to be there when you need it. And, because it uses standard command-line conventions (such as writing to STDOUT, which can be redirected), tcpdump can be used in all sorts of creative, interesting and extremely useful ways.

In this article, I introduce some of the basics of packet capture and provide a breakdown of tcpdump syntax and usage. I show how to use tcpdump to zero in on specific packets and reveal the useful information they contain. I provide some real-world examples of how tcpdump can help put the details of what’s happening on your network at your fingertips, and why tcpdump is still a must-have in any admin’s toolbox.
Essential Concepts

Before you can begin to master tcpdump, you should understand some of the fundamentals that apply to using all packet sniffers:

Packet capturing is passive—it doesn’t transmit or alter network traffic.

You can capture only the packets that your system receives. On a typical switched network, that excludes unicast traffic between other hosts (packets not sent to or from your machine).

You can capture only packets addressed to your system, unless the network interface is in promiscuous mode.

It is assumed that you’re interested in seeing more than just your local traffic, so tcpdump turns on promiscuous mode automatically (which requires root privileges). But, in order for your network card to receive the packets in the first place, you still have to be where the traffic is, so to speak.
Anatomy of a tcpdump Command

A tcpdump command consists of two parts: a set of options followed by a filter expression:

tcpdump -i eth0 -vvn icmp, udp or tcp

So here we are using the options -i eth0 -vvn with the expression of icmp, udp or tcp (choose one).

The expression identifies which packets to capture, and the options define, in part, how those packets are displayed as well as other aspects of program behavior.

Options

tcpdump options follow the standard command-line flag/switch syntax conventions. Some flags accept a parameter, such as -i to specify the capture interface, while others are standalone switches and can be clustered, such as -v to increase verbosity and -n to turn off name resolution.

The man page for tcpdump lists all available options, but here are a few of the noteworthy ones:

-i interface: interface to listen on.
-v, -vv, -vvv: more verbose.
-q: less verbose.
-e: print link-level (Ethernet) headers.
-N: display relative hostnames.
-t: don’t print timestamps.
-n: disable name lookups.
-s0 (or -s 0): use the max “snaplen”—capture full packets (default in recent versions of tcpdump).

None of these are required. User-supplied options simply modify the default program behavior, which is to capture from the first interface, and then print descriptions of matching packets on the screen in a single-line format.

Filter Expression

The filter expression is the Boolean (true or false) criteria for “matching” packets. All packets that do not match the expression are ignored.

The filter expression syntax is robust and flexible. It consists primarily of keywords called primitives, which represent various packet-matching qualifiers, such as protocol, address, port and direction. These can be chained together with and/or, grouped and nested with parentheses, and negated with not to achieve virtually any criteria.

Because the primitives have friendly names and do a lot of the heavy lifting, filter expressions are generally self-explanatory and easy to read and construct. The syntax is fully described in the pcap-filter man page, but here are a few example filter expressions:

tcp
port 25 and not host 10.0.0.3
icmp or arp or udp
vlan 3 and ether src host aa:bb:cc:dd:ee:ff
arp or udp port 53
icmp and $dst host mrsmith or dst host mrpink$

Like the options, filter expressions are not required. An empty filter expression simply matches all packets.

Understanding tcpdump Output

How much sense the output makes depends on how well you understand the protocols in question. tcpdump tailors its output to match the protocol(s) of the given packet.

For example, ARP packets are displayed like this when tcpdump is called with -t and -n (timestamps and name lookups turned off):

arp who-has 10.0.0.1 tell 10.0.0.2
arp reply 10.0.0.1 is-at 00:01:02:03:04:05

ARP is a simple protocol used to resolve IPs into MAC addresses. As you can see above, tcpdump describes these packets in a correspondingly simple format. DNS packets, on the other hand, are displayed completely different:

IP 10.0.0.2.50435 &gt; 10.0.0.1.53: 19+ A? example.com. (34)
IP 10.0.0.1.53 &gt; 10.0.0.2.50435: 19 1/0/0 A 10.10.10.198 (50)

This may seem cryptic at first, but it makes more sense when you understand how protocol layers work. DNS is a more complicated protocol than ARP to begin with, but it also operates on a higher layer. This means it runs over top of other lower-level protocols, which also are displayed in the output.

Unlike ARP, which is a non-routable, layer-3 protocol, DNS is an Internet-wide protocol. It relies on UDP and IP to carry and route it across the Internet, which makes it a layer-5 protocol (UDP is layer-4, and IP is layer-3).

The underlying UDP/IP information, consisting of the source and destination IP/port, is displayed on the left side of the colon, followed by the remaining DNS-specific information on the right.

Even though this DNS information still is displayed in a highly condensed format, you should be able to recognize the essential elements if you know the basics of DNS. The first packet is a query for example.com, and the second packet is an answer, giving the address 10.10.10.198. These are the kind of packets that are generated from simple DNS lookups.

See the OUTPUT FORMAT section of the tcpdump man page for complete descriptions of all the supported protocol-specific output formats. Some protocols are better served than others by their output format, but I’ve found that tcpdump does a pretty good job in general of showing the most useful information about a given protocol.

In addition to its normal behavior of printing packet descriptions to the screen, tcpdump also supports a mode of operation where it writes packets to a file instead. This mode is activated when the -w option is used to specify an output capture file.

When writing to a file, tcpdump uses a completely different format from when it writes to the screen. When writing to the screen, formatted text descriptions of packets are printed. When writing to a file, the raw packets are recorded as is, without analysis.

Instead of doing a live capture, tcpdump also can read from an existing capture file as input with the -r option. Because tcpdump capture files use the universally supported “pcap” format, they also can be opened by other applications, including Wireshark.

This gives you the option to capture packets with tcpdump on one host, but perform analysis on a different host by transferring and loading the capture file. This lets you use Wireshark on your local workstation without having to attach it to the network and location you need to capture from.

Analyzing TCP-Based Application Protocols

tcpdump is a packet-based analyzer, and it works great for connectionless, packet-based protocols like IP, UDP, DHCP, DNS and ICMP. However, it cannot directly analyze “connection-oriented” protocols, such as HTTP, SMTP and IMAP, because they work completely different.

They do not have the concept of “packets”. Instead, they operate over the stream-based connections of TCP, which provide an abstracted communications layer. These application protocols are really more like interactive console programs than packet-based network protocols.

TCP transparently handles all of the underlying details required to provide these reliable, end-to-end, session-style connections. This includes encapsulating the stream-based data into packets (called segments) that can be sent across the network. All of these details are hidden below the application layer.

In order to capture TCP-based application protocols, an extra step is needed beyond capturing packets. Because each TCP segment is only a slice of application data, it can’t be used individually to obtain any meaningful information. You first must reassemble TCP sessions (or flows) from the combined sets of individual segments/packets. The application protocol data is contained directly within the sessions.

tcpdump doesn’t have an option to assemble TCP sessions from packets directly, but you can “fake” it by using what I call “the tcpdump strings trick”.

The tcpdump Strings Trick

Usually when I’m capturing traffic, it’s just for the purpose of casual analysis. The data doesn’t need to be perfect if it shows me what I’m looking for and helps me gain some insight.

In these cases, speed and convenience reign supreme. The following trick is along these lines and is one of my favorite tcpdump techniques. It works because:

TCP segments usually are sent in chronological order.
Text-based application protocols produce TCP segments with text payloads.
The data surrounding the text payloads, such as packet headers, is usually not text.
The UNIX command strings filters out binary data from streams preserving only text (printable characters).
When tcpdump is called with -w - it prints raw packets to STDOUT.

Put it all together, and you get a command that dumps real-time HTTP session data:

tcpdump -l -s0 -w - tcp dst port 80 | strings

The -l option above turns on line buffering, which makes sure data gets printed to the screen right away.

What is happening here is tcpdump is printing the raw, binary data to the screen. This uses a twist on the -w option where the special filename - writes to STDOUT instead of a file. Normally, doing this would display all kinds of gibberish, but that’s where the strings command comes in, it allows only data recognized as text through to the screen.

There are few caveats to be aware of.

First, data from multiple sessions received simultaneously is displayed simultaneously, clobbering your output. The more refined you make the filter expression, the less of a problem this will be. You also should run separate commands (in separate shells) for the client and server side of a session:

tcpdump -l -s0 -w - tcp dst port 80 | strings
tcpdump -l -s0 -w - tcp src port 80 | strings

Also, you should expect to see a few gibberish characters here and there whenever a sequence of binary data also happens to look like text characters. You can cut down on this by increasing min-len (see the strings man page).

This trick works just as well for other text-based protocols.

HTTP and SMTP Analysis

Using the strings trick in the previous section, you can capture HTTP data even though tcpdump doesn’t actually understand anything about it. You then can “analyze” it further in any number of ways.

If you wanted to see all the Web sites being accessed by “mypc” in real time, for example, you could run this command on the firewall (assume the internal interface is eth0):

tcpdump -i eth0 -l -s0 -w - host mypc and port 80 | strings | grep 'GET\|Host'

In this example, I’m using a simple grep command to display only lines with GET or Host. These strings show up in HTTP requests and together show the URLs being accessed.

This works just as well for SMTP. You could run this on your mail server to watch e-mail senders and recipients:

tcpdump -l -s0 -w - tcp dst port 25 | strings | grep -i 'MAIL FROM\|RCPT TO'

These are just a few silly examples to illustrate what’s possible. You obviously could take it beyond grep. You could go as far as to write a script to do all sorts of sophisticated things. You probably wouldn’t take that too far, however, because at that point, there are better tools that actually are designed to do that sort of thing.

The real value of tcpdump is the ability to do these kinds of things interactively and on a whim. It’s the power to look inside any aspect of your network whenever you want without a lot of effort.

Debugging Routes and VPN Links

tcpdump is really handy when debugging VPNs and other network connections by showing where packets are showing up and where they aren’t. Let’s say you’ve set up a standard routable network-to-network VPN between 10.0.50.0/24 and 192.168.5.0/24

network topology

If it’s operating properly, hosts from either network should be able to ping one another. However, if you are not getting replies when pinging host D from host A, for instance, you can use tcpdump to zero in on exactly where the breakdown is occurring:

tcpdump -tn icmp and host 10.0.50.2

In this example, during a ping from 10.0.50.2 to 192.168.5.38, each round trip should show up as a pair of packets like the following, regardless of from which of the four systems the tcpdump command is run:

IP 10.0.50.2 &gt; 192.168.5.38: ICMP echo request, id 46687, seq 1, length 64
IP 192.168.5.38 &gt; 10.0.50.2: ICMP echo reply, id 46687, seq 1, length 64

If the request packets make it to host C (the remote gateway) but not to D, this indicates that the VPN itself is working, but there could be a routing problem. If host D receives the request but doesn’t generate a reply, it probably has ICMP blocked. If it does generate a reply but it doesn’t make it back to C, then D might not have the right gateway configured to get back to 10.0.50.0/24.

Using tcpdump, you can follow the ping through all eight possible points of failure as it makes its way across the network and back again.

Conclusion

I hope this article has piqued your interest in tcpdump and given you some new ideas. Hopefully, you also enjoyed the examples that barely scratch the surface of what’s possible.

Besides its many built-in features and options, as I showed in several examples, tcpdump can be used as a packet-data conduit by piping into other commands to expand the possibilities further—even if you do manage to exhaust its extensive “advertised” capabilities. The ways to use tcpdump are limited only by your imagination.

tcpdump also is an incredible learning tool. There is no better way to learn how networks and protocols work than from watching their actual packets.

Don’t forget to check out the tcpdump and pcap-filter man pages for additional details and information.

tcpdump has been the de facto packet capture tool for the past 25 years. It really did spawn the whole genre of network utilities based on sniffing and analyzing packets. Prior to tcpdump, packet capture had such high processing demands that it was largely impractical. But tcpdump introduced some key innovations and optimizations that helped make packet capture more viable, for all systems both for regular systems and for networks with a large amounts of traffic.

a simple network listener

This is just a simple c program I use to test that I can a) open a port and b) connect to that port and what the server sees me as. To compile simply use gcc server.c -o server. Be sure to edit the code to change the port to the one you want to listen on/test. Then simply telnet to the host and port from the machine you want to test. The server will report the ip it sees.



/*
** server.c -- simple port listener used for testing.
*/

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <netdb.h>
#include <arpa/inet.h>
#include <sys/wait.h>
#include <signal.h>

#define PORT "1521"  // the port users will be connecting to

#define BACKLOG 10     // how many pending connections queue will hold

void sigchld_handler(int s)
{
    while(waitpid(-1, NULL, WNOHANG) > 0);
}

// get sockaddr, IPv4 or IPv6:
void *get_in_addr(struct sockaddr *sa)
{
    if (sa->sa_family == AF_INET) {
        return &(((struct sockaddr_in*)sa)->sin_addr);
    }

    return &(((struct sockaddr_in6*)sa)->sin6_addr);
}

int main(void)
{
    int sockfd, new_fd;  // listen on sock_fd, new connection on new_fd
    struct addrinfo hints, *servinfo, *p;
    struct sockaddr_storage their_addr; // connector's address information
    socklen_t sin_size;
    struct sigaction sa;
    int yes=1;
    char s[INET6_ADDRSTRLEN];
    int rv;

    memset(&hints, 0, sizeof hints);
    hints.ai_family = AF_UNSPEC;
    hints.ai_socktype = SOCK_STREAM;
    hints.ai_flags = AI_PASSIVE; // use my IP

    if ((rv = getaddrinfo(NULL, PORT, &hints, &servinfo)) != 0) {
        fprintf(stderr, "getaddrinfo: %s\n", gai_strerror(rv));
        return 1;
    }

    // loop through all the results and bind to the first we can
    for(p = servinfo; p != NULL; p = p->ai_next) {
        if ((sockfd = socket(p->ai_family, p->ai_socktype,
                p->ai_protocol)) == -1) {
            perror("server: socket");
            continue;
        }

        if (setsockopt(sockfd, SOL_SOCKET, SO_REUSEADDR, &yes,
                sizeof(int)) == -1) {
            perror("setsockopt");
            exit(1);
        }

        if (bind(sockfd, p->ai_addr, p->ai_addrlen) == -1) {
            close(sockfd);
            perror("server: bind");
            continue;
        }

        break;
    }

    if (p == NULL)  {
        fprintf(stderr, "server: failed to bind\n");
        return 2;
    }

    freeaddrinfo(servinfo); // all done with this structure

    if (listen(sockfd, BACKLOG) == -1) {
        perror("listen");
        exit(1);
    }

    sa.sa_handler = sigchld_handler; // reap all dead processes
    sigemptyset(&sa.sa_mask);
    sa.sa_flags = SA_RESTART;
    if (sigaction(SIGCHLD, &sa, NULL) == -1) {
        perror("sigaction");
        exit(1);
    }

    printf("server: waiting for connections...\n");

    while(1) {  // main accept() loop
        sin_size = sizeof their_addr;
        new_fd = accept(sockfd, (struct sockaddr *)&their_addr, &sin_size);
        if (new_fd == -1) {
            perror("accept");
            continue;
        }

        inet_ntop(their_addr.ss_family,
            get_in_addr((struct sockaddr *)&their_addr),
            s, sizeof s);
        printf("server: got connection from %s\n", s);

        if (!fork()) { // this is the child process
            close(sockfd); // child doesn't need the listener
            if (send(new_fd, "You are connected!\n  ", 13, 0) == -1)
                perror("send");
            close(new_fd);
            exit(0);
        }
        close(new_fd);  // parent doesn't need this
    }

    return 0;
}

My special .bashrc setup

I get a lot of questions about my short cuts. Here is my .bashrc I use so people have some examples of things to do with their profile. I put a description of what it does in this format so it is easy to understand. I made sure to comment them out so you can just copy paste this and go with it. I will post updates as I get the urge or requests.


# .bashrc
# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi

# This just setups up the colors for you. I stole this from the Red Hat init system


# Define some colors first:
red='\e[0;31m'
RED='\e[1;31m'
blue='\e[0;34m'
BLUE='\e[1;34m'
cyan='\e[0;36m'
CYAN='\e[1;36m'
GREEN='\e[0;32'
YELLOW='\e[1;33'
NC='\e[0m'              # No Color

#Some short cut commands I use on a regular basis


# User specific aliases and functions
#-------------------------------------------------------------
# The 'ls' family
#-------------------------------------------------------------
alias ll="ls -l --group-directories-first"
alias la='ls -Al'          # show hidden files
alias lx='ls -lXB'         # sort by extension
alias lk='ls -lSr'         # sort by size, biggest last
alias lc='ls -ltcr'        # sort by and show change time, most recent last
alias lu='ls -ltur'        # sort by and show access time, most recent last
alias lt='ls -ltr'         # sort by date, most recent last
alias lm='ls -al |more'    # pipe through 'more'
alias lr='ls -lR'          # recursive ls
alias tree='tree -Csu'     # nice alternative to 'recursive ls'
alias h='history'
alias j='jobs -l'
alias which='type -a'
alias ..='cd ..'
alias du='du -kh'       # Makes a more readable output.
alias dk='df -kTh'
alias lot='lsof -i tcp'    #List all open files using tcp
alias lou='lsof -i tcp'    #List all open files using udp

#This should be obvious; but it is the help function I wrote since I forget sometimes what I write


#-----------------------------------------------------------
# Functions - Handy functions to have
#-----------------------------------------------------------
function helpme()
{
    echo -e "\n"
    echo -e "\t${BLUE}List of Commands and Aliases$NC"
    echo -e "\n"
    echo -e "\tAliases"
    echo -e "--------------------------------------------------------------------"
        echo -e "${RED}ll$NC    \t\tshow listing with directories first"
        echo -e "${RED}la$NC    \t\tshow hidden files"
        echo -e "${RED}lx$NC    \t\tsort by extension"
        echo -e "${RED}lk$NC    \t\tsort by size, biggest last"
        echo -e "${RED}lc$NC    \t\tsort by and show change time, most recent last"
        echo -e "${RED}lu$NC    \t\tsort by and show access time, most recent last"
        echo -e "${RED}lt$NC    \t\tsort by date, most recent last"
        echo -e "${RED}lm$NC    \t\tpipe through 'more'"
        echo -e "${RED}lr$NC    \t\trecursive ls"
        echo -e "${RED}tree$NC    \t\tnice alternative to 'recursive ls'"
        echo -e "${RED}h$NC    \t\thistory"
    echo -e "${RED}j$NC    \t\tlist jobs"
    echo -e "${RED}..$NC    \t\tcd .."
    echo -e "${RED}du$NC    \t\tmakes format more readable"
    echo -e "${RED}dk$NC    \t\teasier format and more information"
        echo -e "${RED}lot$NC   \t\tList all open files using tcp"
        echo -e "${RED}lou$NC    \t\tList all open files using udp"
    echo -e "\n"
    echo -e "\tCommands"
    echo -e "---------------------------------------------------------------------"
    echo -e "extract <filename>    \t\textract any archive"
    echo -e "change2user <username>    \t\tsudo to a user and setup X for remote display"
    echo -e "change2root         \t\tsudo to root with X display setup for remote export"
    echo -e "lowercase        \t\tmove filenames to lowercase"
    echo -e "killps            \t\tkill process by name"
    echo -e "repeat    <N> <command>    \t\trepeat command N times"
    echo -e "corename <core>    \t\tGet name of app that created a corefile"
    echo -e "ii            \t\tGet information about current host"
    echo -e "\n"
}

#Command to extract any kind of archive file quick and easy


function extract()      # Handy Extract Program.
{
     if [ -f $1 ] ; then
         case $1 in
             *.tar.bz2)   tar xvjf $1     ;;
             *.tar.gz)    tar xvzf $1     ;;
             *.bz2)       bunzip2 $1      ;;
             *.rar)       unrar x $1      ;;
             *.gz)        gunzip $1       ;;
             *.tar)       tar xvf $1      ;;
             *.tbz2)      tar xvjf $1     ;;
             *.tgz)       tar xvzf $1     ;;
             *.zip)       unzip $1        ;;
             *.Z)         uncompress $1   ;;
             *.7z)        7z x $1         ;;
             *)           echo "'$1' cannot be extracted via >extract<" ;;
         esac
     else
         echo "'$1' is not a valid file"
     fi
}

#The next to are so I can change (sudo) to a user first, root second and export the display


function change2user() #Handy for exporting displays as a different user than the one you logged in as
{
     echo "Changing permissions on Xauthority file."
     chmod 644 ~/.Xauthority
     echo -n "Getting user shell...."
     NEWUSERSHELL=`ypcat passwd | grep $1 | awk -F: '{ print $7 }'`
     echo $NEWUSERSHELL
     XAUTHORITY=$HOME/.Xauthority DISPLAY=$DISPLAY sudo su - $1 $NEWUSERSHELL
}

function change2root() #Handy for exporting the display when you su - root
{
     echo "Changing permissions on Xauthority file."
     chmod 644 ~/.Xauthority
     echo -n "Getting user shell...."
     NEWUSERSHELL=/bin/bash
     echo $NEWUSERSHELL
     XAUTHORITY=$HOME/.Xauthority DISPLAY=$DISPLAY sudo su -
}

#I hate uppercase file names; this fix those for me by changing them all to lowercase.


function lowercase()  # move filenames to lowercase
{
    for file ; do
        filename=${file##*/}
        case "$filename" in
        */*) dirname==${file%/*} ;;
        *) dirname=.;;
        esac
        nf=$(echo $filename | tr A-Z a-z)
        newname="${dirname}/${nf}"
        if [ "$nf" != "$filename" ]; then
            mv "$file" "$newname"
            echo "lowercase: $file --> $newname"
        else
            echo "lowercase: $file not changed."
        fi
    done
}

#Kill a process by name quick and easy. No Fuss, no Mess


function killps()                 # Kill by process name.
{
    local pid pname sig="-TERM"   # Default signal.
    if [ "$#" -lt 1 ] || [ "$#" -gt 2 ]; then
        echo "Usage: killps [-SIGNAL] pattern"
        return;
    fi
    if [ $# = 2 ]; then sig=$1 ; fi
    for pid in $(my_ps| awk '!/awk/ && $0~pat { print $1 }' pat=${!#} ) ; do
        pname=$(my_ps | awk '$1~var { print $5 }' var=$pid )
        if ask "Kill process $pid <$pname> with signal $sig?"
            then kill $sig $pid
        fi
    done
}

#This was an old command, but gives me my ip (used by one of my firewall scripts)


function my_ip() # Get IP adresses.
{
    MY_IP=$(/sbin/ifconfig eth0 | awk '/inet/ { print $2 } ' | \
sed -e s/addr://)
    MY_ISP=$(/sbin/ifconfig eth0 | awk '/P-t-P/ { print $3 } ' | \
sed -e s/P-t-P://)
}

#This gives me my local hosts information. I use this as part of a login script to confirm where I am.


function ii()   # Get current host related info.
{
    echo -e "\nYou are logged on ${RED}$HOST"
    echo -e "\nAdditionnal information:$NC " ; uname -a
    echo -e "\n${RED}Users logged on:$NC " ; w -h
    echo -e "\n${RED}Current date :$NC " ; date
    echo -e "\n${RED}Machine stats :$NC " ; uptime
    echo -e "\n${RED}Memory stats :$NC " ; free
    my_ip 2>&- ;
    echo -e "\n${RED}Local IP Address :$NC" ; echo ${MY_IP:-"Not connected"}
    echo -e "\n${RED}ISP Address :$NC" ; echo ${MY_ISP:-"Not connected"}
    echo -e "\n${RED}Open connections :$NC "; netstat -pan --inet;
    echo
}

#This just repeats command(s) X times, useful in some situations


function repeat()       # Repeat n times command.
{
    local i max
    max=$1; shift;
    for ((i=1; i <= max ; i++)); do  # --> C-like syntax
        eval "$@";
    done
}

Just used by other functions


function ask()          # See 'killps' for example of use.
{
    echo -n "$@" '[y/n] ' ; read ans
    case "$ans" in
        y*|Y*) return 0 ;;
        *) return 1 ;;
    esac
}

#This helps you find out what created those core files you find, comes in handy


function corename()   # Get name of app that created a corefile.
{
    for file ; do
        echo -n $file : ; gdb --core=$file --batch | head -1
    done
}

The Mad Hatter

Pages

Categories

Archives

Sponsored Links

Resize Linux storage with LVM

Linux Tips

tcpdump tips

Options

Filter Expression

Understanding tcpdump Output

Analyzing TCP-Based Application Protocols

The tcpdump Strings Trick

HTTP and SMTP Analysis

Debugging Routes and VPN Links

Conclusion

a simple network listener

My special .bashrc setup

Links

Login

Sponsored Links

Sponsored Links