### Linux modprobe Command Examples to View, Install, Remove Modules ###
modprobe utility is used to add loadable modules to the Linux kernel. You can also view and remove modules using modprobe command.
1. List Available Kernel Modules
modprobe -l will display all available modules as shown below.
2. List Currently Loaded Modules
While the above modprobe command shows all available modules, lsmod command will display all modules that are currently loaded in the Linux kernel.
lsmod | less
3. Install New modules into Linux Kernel
In order to insert a new module into the kernel, execute the modprobe command with the module name.
Example: sudo modprobe vmhgfs ; lsmod | grep vmhgfs
4. Load New Modules with the Different Name to Avoid Conflicts
Consider, in some cases you are supposed to load a new module but with the same module name another module got already loaded for different purposes.
To load a module with a different name, use the modprobe option -o as shown below.
- sudo modprobe vmhgfs -o vm_hgfs
5. Remove the Currently Loaded Module
If you’ve loaded a module to Linux kernel for some testing purpose, you might want to unload (remove) it from the kernel.
- modprobe -r vmhgfs
Ethtool utility is used to view and change the ethernet device parameters.
* Full duplex : Enables sending and receiving of packets at the same time. This mode is used when the ethernet device is connected to a switch.
* Half duplex : Enables either sending or receiving of packets at a single point of time. This mode is used when the ethernet device is connected to a hub.
* Auto-negotiation : If enabled, the ethernet device itself decides whether to use either full duplex or half duplex based on the network the ethernet device attached to.
### Troubleshooting Using dmesg Command in Unix and Linux ###
During system bootup process, kernel gets loaded into the memory and it controls the entire system.
When the system boots up, it prints number of messages on the screen that displays information about the hardware devices that the kernel detects during boot process.
These messages are available in kernel ring buffer and whenever the new message comes the old message gets overwritten
1. View the Boot Messages
By executing the dmesg command, you can view the hardwares that are detected during bootup process and it’s configuration details. There are lot of useful information displayed in dmesg. Just browse through them line by line and try to understand what it means. Once you have an idea of the kind of messages it displays, you might find it helpful for troubleshooting, when you encounter an issue.
# dmesg | more
2. View Available System Memory
# dmesg | grep Memory
3. View Ethernet Link Status (UP/DOWN)
# dmesg | grep eth
4. Change the dmesg Buffer Size in /boot/config- file
Linux allows to you change the default size of the dmesg buffer. The CONFIG_LOG_BUF_SHIFT parameter in the /boot/config-2.6.18-194.el5 file (or similar file on your system) can be changed to modify the dmesg buffer.
# grep CONFIG_LOG_BUF_SHIFT /boot/config-`uname -r`
5. Clear Messages in dmesg Buffer
# dmesg -c
6. dmesg timestamp: Date and Time of Each Boot Message in dmesg
By default the dmesg don’t have the timestamp associated with them. However Linux provides a way to see the date and time for each boot messages in dmesg in the /var/log/kern.log file as shown below.
klogd service should be enabled and configured properly to log the messages in /var/log/kern.log file.
# dmesg | grep "L2 cache"
# grep "L2 cache" kern.log.1
### Explore Linux /proc File System (/proc directories, /proc files) ###
Inside the /proc directory, you’ll see two types of content — numbered directories, and system information files.
/proc is not a real file system, it is a virtual file system. For example, if you do ls -l /proc/stat, you’ll notice that it has a size of 0 bytes, but if you do “cat /proc/stat”, you’ll see some content inside the file.
1. /proc Directories with names as numbers
Do a ls -l /proc, and you’ll see lot of directories with just numbers. These numbers represents the process ids, the files inside this numbered directory corresponds to the process with that particular PID.
Following are the important files located under each numbered directory (for each process):
* cmdline – command line of the command.
* environ – environment variables.
* fd – Contains the file descriptors which is linked to the appropriate files.
* limits – Contains the information about the specific limits to the process.
* mounts – mount related information
Following are the important links under each numbered directory (for each process):
* cwd – Link to current working directory of the process.
* exe – Link to executable of the process.
* root – Link to the root directory of the process.
2. /proc Files about the system information
Following are some files which are available under /proc, that contains system information such as cpuinfo, meminfo, loadavg.
* /proc/cpuinfo – information about CPU,
* /proc/meminfo – information about memory,
* /proc/loadvg – load average,
* /proc/partitions – partition related information,
* /proc/version – linux version
Some Linux commands read the information from this /proc files and displays it. For example, free command, reads the memory information from /proc/meminfo file, formats it, and displays it.
# /proc/cmdline – Kernel command line
# /proc/cpuinfo – Information about the processors.
# /proc/devices – List of device drivers configured into the currently running kernel.
# /proc/dma – Shows which DMA channels are being used at the moment.
# /proc/fb – Frame Buffer devices.
# /proc/filesystems – File systems supported by the kernel.
# /proc/interrupts – Number of interrupts per IRQ on architecture.
# /proc/iomem – This file shows the current map of the system’s memory for its various devices
# /proc/ioports – provides a list of currently registered port regions used for input or output communication with a device
# /proc/loadavg – Contains load average of the system
The first three columns measure CPU utilization of the last 1, 5, and 10 minute periods.
The fourth column shows the number of currently running processes and the total number of processes.
The last column displays the last process ID used.
# /proc/locks – Displays the files currently locked by the kernel
Sample line:
1: POSIX ADVISORY WRITE 14375 08:03:114727 0 EOF
# /proc/meminfo – Current utilization of primary memory on the system
# /proc/misc – This file lists miscellaneous drivers registered on the miscellaneous major device, which is number 10
# /proc/modules – Displays a list of all modules that have been loaded by the system
# /proc/mounts – This file provides a quick list of all mounts in use by the system
# /proc/partitions – Very detailed information on the various partitions currently available to the system
# /proc/pci – Full listing of every PCI device on your system
# /proc/stat – Keeps track of a variety of different statistics about the system since it was last restarted
# /proc/swap – Measures swap space and its utilization
# /proc/uptime – Contains information about uptime of the system
# /proc/version – Version of the Linux kernel, gcc, name of the Linux flavor installed.
##How to Convert DEB to RPM (RPM to DEB) Package Using Alien Command##
1. Convert RPM to DEB
# sudo apt-get install alien
Use alien command to convert rpm to deb file
The following example converts the linuxconf-devel rpm file to linuxconf-devel deb file. Once you generate the deb file, you can install it on Ubuntu or Debian.
# alien linuxconf-devel-1.16r10-2.i386.rpm linuxconf-devel_1.16r10-3_i386.deb generated
2. Convert DEB to RPM
Use alien to convert deb to rpm file
Use alient -r option to convert a deb file to rpm file. The following example converts libsox deb file to libsox rpm file. Once you generate the rpm file, you can install it on Red Hat, or CentOS.
# sudo alien -r libsox1_14.2.0-1_i386.deb libsox1-14.2.0-2.i386.rpm generated
3. Convert to SLP, LSB, Slackware TGZ packages
You can also use alien command to convert files to Stampede slp package, LSB package, and Slackware tgz package. Do alien -h to see available options.
Disable - 1 / Enable - 0, ping reply Temporarily
# echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_all
Disable ping reply Permanently
Edit the sysctl.conf file and add the following line.
net.ipv4.icmp_echo_ignore_all = 1
Execute sysctl -p to enforce this setting immediately.
# sysctl -p
netcat usage:-
target: netcat -l -p 1234 > vx64-datacenter_1.5.1.6.iso
source: cat vx64-datacenter_1.5.1.6.iso | netcat opsvx64 1234 -q 10
Groff stands for GNU troff. Groff is typically used to create man pages. But, it does mote than that. Its an old school utility which has been around for almost 20 years but still holds its presence
Objdump command in linux is used to provide thorough information on object files. This command is maily used by the programmers who work on compilers, but still its a very handly tool for normal programmers also when it comes to debugging
SQLite
SQLite3 is very lightweight SQL database which focuses on simplicity more than anything else. This is a self-contained serverless database engine, which is very simple to install and use.
While most of the commands in the SQLite are similar to SQL commands of other databases like
MySQL and ORACLE, there are some SQLite SQL commands that are different.
A SQLite database is nothing but a file that gets created.
#sqlite3 company.db
#Rename a Table
sqlite> alter table department rename to dept;
#Add a Column to an Existing Table
sqlite> alter table employee add column deptid integer;
#View all Tables in Database
sqlite>.tables
#Create an Index
creates an unique index called empidx on the empid field of employee table
sqlite> create unique index empidx on employee(empid);
#Create a Trigger
First add a data column called "updateon" on employee table
sqlite> alter table employee add column updatedon date;
Next, create a file that has the trigger definition. The following trigger will update the "updatedon" date column with the current timestamp whenever you perform an update on this table.
#vi employee_update_trg.sql
#sqlite3 company.db < employee_update_trg.sql
#Create a View
The following example creates a view called "empdept" which combines fields from both employee and dept table.
sqlite> create view empdept as select empid, e.name, title, d.name, location from employee e, dept d where e.deptid = d.deptid;
#Explain Query Plan
Execute "explain query plan", to get information about the table that is getting used in a query or view. This is very helpful when you are debugging a complex query with multiple joins
on several tables;
sqlite> explain query plan select * from empdept;
#Attach and Detach Database
When you have multiple database, you can use attach command to execute queries across database
For example, if you have two database that has the same table name with different data, you can create a union query across the database to view the combined records
sqlite> attach database 'company1.db' as c1;
sqlite> attach database 'company2.db' as c2;
sqlite> detach c1;
Strip command is used mostly in situations where you want to produce a production quality object file which contains minimum required information so that is can be light weight. You can also use it if you don't want your exectuable or object file to get reverse engineered
SNMP stands for Simple Network Management protocol
This protocol started becoming popular and it forms the basis of network management today
Through snmp one can retrieve information about network devices like routers,printers,hubs or even normal computers. The information that can be retrieved through snmp is endless.
Examples
- System up time
- CPU usage level
- Disk usage level
- Network settings etc
SNMP system design
A computer network system that uses SNMP for network management consists of the three fundamental componenets
1. The SNMP manager: It is a software that usually runs on the machine of network administrator or any human manager managing the computer network
2. The SNMP agent: It is a software that usually runs on the network node that is to be monitored. This node could be a printer. router etc
3. The SNMP MIB: MIB stands for management information base. This component makes sure that the data exchange between the manager and the agent remains structured
SNMP uses UDP
At the transport layer, the protocol used for SNMP message transportation is UDP. This is because UDP outperforms TCP in lossy networks where congestion is usually very high. One thing to remember is to fine tune the time-outs of UDP to fetch the best performance in lossy networks
UNAME
$uname -s (Which displays the kernel name)
Linux
$uname -m (Machine hardware name)
x86_64
$uname -o (Operating System name )
GNU/Linux
Octal Dump
od command in Linux is used to output the contents of a file in different formats with the octal format being the default
Hexadecimal (using -x along with -A)
Octal (using -o along with -A)
Decimal (using -d along with -A)
lsof (Identify Open Files)
lsof stands for List Open Files. It is a command line utility which is used to list the information about the files that are opened by various processes. In unix, everything is a file (pipes,sockets,directories,devices etc). So by using lsof, you can get the information about any opened files.
- List processes which opened a specific file
You can list only the processes which opened a specific file, by providing the filename as arguments
#lsof /var/log/syslog
- List openned files under a directory
You can list the processes which opened files under a specified directory using '+D' option.
#lsof +D /var/log/
- List openned files based on process names starting with
#lsof -c ssh -c init
- List processes using a mount point
#lsof /home
#lsof +D /home/
- List files opened by a specific user
In order to find the list of files opened by a specific users, use '-u' option
#lsof -u gyani
- List all open files by a specific process
#lsof -p 1753
- Kill all process that belongs to a particular user
#kill -9 `lsof -t -u gyani`
- List processes which are listening on a particular port
#lsof -i :25
- List all TCP or UDP connections
#lsof -i tcp; lsof -i udp;
Bash Tweaks
#sed '$aEND-OF-FILE' yourFile # Add a word to everyline of the file at the end of the line
#sed -i 's/^/This /' infile or sed 's/^/This /' infile # Add a word to everyline of the file at the begining of the line
#sed "s/$/,/g" filename #At the end of everyline I need to add a comma at the end of the file
Use sed where ^ is a regular expression, signifying the beginning of the line
modprobe utility is used to add loadable modules to the Linux kernel. You can also view and remove modules using modprobe command.
1. List Available Kernel Modules
modprobe -l will display all available modules as shown below.
2. List Currently Loaded Modules
While the above modprobe command shows all available modules, lsmod command will display all modules that are currently loaded in the Linux kernel.
lsmod | less
3. Install New modules into Linux Kernel
In order to insert a new module into the kernel, execute the modprobe command with the module name.
Example: sudo modprobe vmhgfs ; lsmod | grep vmhgfs
4. Load New Modules with the Different Name to Avoid Conflicts
Consider, in some cases you are supposed to load a new module but with the same module name another module got already loaded for different purposes.
To load a module with a different name, use the modprobe option -o as shown below.
- sudo modprobe vmhgfs -o vm_hgfs
5. Remove the Currently Loaded Module
If you’ve loaded a module to Linux kernel for some testing purpose, you might want to unload (remove) it from the kernel.
- modprobe -r vmhgfs
Ethtool utility is used to view and change the ethernet device parameters.
* Full duplex : Enables sending and receiving of packets at the same time. This mode is used when the ethernet device is connected to a switch.
* Half duplex : Enables either sending or receiving of packets at a single point of time. This mode is used when the ethernet device is connected to a hub.
* Auto-negotiation : If enabled, the ethernet device itself decides whether to use either full duplex or half duplex based on the network the ethernet device attached to.
### Troubleshooting Using dmesg Command in Unix and Linux ###
During system bootup process, kernel gets loaded into the memory and it controls the entire system.
When the system boots up, it prints number of messages on the screen that displays information about the hardware devices that the kernel detects during boot process.
These messages are available in kernel ring buffer and whenever the new message comes the old message gets overwritten
1. View the Boot Messages
By executing the dmesg command, you can view the hardwares that are detected during bootup process and it’s configuration details. There are lot of useful information displayed in dmesg. Just browse through them line by line and try to understand what it means. Once you have an idea of the kind of messages it displays, you might find it helpful for troubleshooting, when you encounter an issue.
# dmesg | more
2. View Available System Memory
# dmesg | grep Memory
3. View Ethernet Link Status (UP/DOWN)
# dmesg | grep eth
4. Change the dmesg Buffer Size in /boot/config- file
Linux allows to you change the default size of the dmesg buffer. The CONFIG_LOG_BUF_SHIFT parameter in the /boot/config-2.6.18-194.el5 file (or similar file on your system) can be changed to modify the dmesg buffer.
# grep CONFIG_LOG_BUF_SHIFT /boot/config-`uname -r`
5. Clear Messages in dmesg Buffer
# dmesg -c
6. dmesg timestamp: Date and Time of Each Boot Message in dmesg
By default the dmesg don’t have the timestamp associated with them. However Linux provides a way to see the date and time for each boot messages in dmesg in the /var/log/kern.log file as shown below.
klogd service should be enabled and configured properly to log the messages in /var/log/kern.log file.
# dmesg | grep "L2 cache"
# grep "L2 cache" kern.log.1
### Explore Linux /proc File System (/proc directories, /proc files) ###
Inside the /proc directory, you’ll see two types of content — numbered directories, and system information files.
/proc is not a real file system, it is a virtual file system. For example, if you do ls -l /proc/stat, you’ll notice that it has a size of 0 bytes, but if you do “cat /proc/stat”, you’ll see some content inside the file.
1. /proc Directories with names as numbers
Do a ls -l /proc, and you’ll see lot of directories with just numbers. These numbers represents the process ids, the files inside this numbered directory corresponds to the process with that particular PID.
Following are the important files located under each numbered directory (for each process):
* cmdline – command line of the command.
* environ – environment variables.
* fd – Contains the file descriptors which is linked to the appropriate files.
* limits – Contains the information about the specific limits to the process.
* mounts – mount related information
Following are the important links under each numbered directory (for each process):
* cwd – Link to current working directory of the process.
* exe – Link to executable of the process.
* root – Link to the root directory of the process.
2. /proc Files about the system information
Following are some files which are available under /proc, that contains system information such as cpuinfo, meminfo, loadavg.
* /proc/cpuinfo – information about CPU,
* /proc/meminfo – information about memory,
* /proc/loadvg – load average,
* /proc/partitions – partition related information,
* /proc/version – linux version
Some Linux commands read the information from this /proc files and displays it. For example, free command, reads the memory information from /proc/meminfo file, formats it, and displays it.
# /proc/cmdline – Kernel command line
# /proc/cpuinfo – Information about the processors.
# /proc/devices – List of device drivers configured into the currently running kernel.
# /proc/dma – Shows which DMA channels are being used at the moment.
# /proc/fb – Frame Buffer devices.
# /proc/filesystems – File systems supported by the kernel.
# /proc/interrupts – Number of interrupts per IRQ on architecture.
# /proc/iomem – This file shows the current map of the system’s memory for its various devices
# /proc/ioports – provides a list of currently registered port regions used for input or output communication with a device
# /proc/loadavg – Contains load average of the system
The first three columns measure CPU utilization of the last 1, 5, and 10 minute periods.
The fourth column shows the number of currently running processes and the total number of processes.
The last column displays the last process ID used.
# /proc/locks – Displays the files currently locked by the kernel
Sample line:
1: POSIX ADVISORY WRITE 14375 08:03:114727 0 EOF
# /proc/meminfo – Current utilization of primary memory on the system
# /proc/misc – This file lists miscellaneous drivers registered on the miscellaneous major device, which is number 10
# /proc/modules – Displays a list of all modules that have been loaded by the system
# /proc/mounts – This file provides a quick list of all mounts in use by the system
# /proc/partitions – Very detailed information on the various partitions currently available to the system
# /proc/pci – Full listing of every PCI device on your system
# /proc/stat – Keeps track of a variety of different statistics about the system since it was last restarted
# /proc/swap – Measures swap space and its utilization
# /proc/uptime – Contains information about uptime of the system
# /proc/version – Version of the Linux kernel, gcc, name of the Linux flavor installed.
##How to Convert DEB to RPM (RPM to DEB) Package Using Alien Command##
1. Convert RPM to DEB
# sudo apt-get install alien
Use alien command to convert rpm to deb file
The following example converts the linuxconf-devel rpm file to linuxconf-devel deb file. Once you generate the deb file, you can install it on Ubuntu or Debian.
# alien linuxconf-devel-1.16r10-2.i386.rpm linuxconf-devel_1.16r10-3_i386.deb generated
2. Convert DEB to RPM
Use alien to convert deb to rpm file
Use alient -r option to convert a deb file to rpm file. The following example converts libsox deb file to libsox rpm file. Once you generate the rpm file, you can install it on Red Hat, or CentOS.
# sudo alien -r libsox1_14.2.0-1_i386.deb libsox1-14.2.0-2.i386.rpm generated
3. Convert to SLP, LSB, Slackware TGZ packages
You can also use alien command to convert files to Stampede slp package, LSB package, and Slackware tgz package. Do alien -h to see available options.
Disable - 1 / Enable - 0, ping reply Temporarily
# echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_all
Disable ping reply Permanently
Edit the sysctl.conf file and add the following line.
net.ipv4.icmp_echo_ignore_all = 1
Execute sysctl -p to enforce this setting immediately.
# sysctl -p
netcat usage:-
target: netcat -l -p 1234 > vx64-datacenter_1.5.1.6.iso
source: cat vx64-datacenter_1.5.1.6.iso | netcat opsvx64 1234 -q 10
Groff stands for GNU troff. Groff is typically used to create man pages. But, it does mote than that. Its an old school utility which has been around for almost 20 years but still holds its presence
Objdump command in linux is used to provide thorough information on object files. This command is maily used by the programmers who work on compilers, but still its a very handly tool for normal programmers also when it comes to debugging
SQLite
SQLite3 is very lightweight SQL database which focuses on simplicity more than anything else. This is a self-contained serverless database engine, which is very simple to install and use.
While most of the commands in the SQLite are similar to SQL commands of other databases like
MySQL and ORACLE, there are some SQLite SQL commands that are different.
A SQLite database is nothing but a file that gets created.
#sqlite3 company.db
#Rename a Table
sqlite> alter table department rename to dept;
#Add a Column to an Existing Table
sqlite> alter table employee add column deptid integer;
#View all Tables in Database
sqlite>.tables
#Create an Index
creates an unique index called empidx on the empid field of employee table
sqlite> create unique index empidx on employee(empid);
#Create a Trigger
First add a data column called "updateon" on employee table
sqlite> alter table employee add column updatedon date;
Next, create a file that has the trigger definition. The following trigger will update the "updatedon" date column with the current timestamp whenever you perform an update on this table.
#vi employee_update_trg.sql
#sqlite3 company.db < employee_update_trg.sql
#Create a View
The following example creates a view called "empdept" which combines fields from both employee and dept table.
sqlite> create view empdept as select empid, e.name, title, d.name, location from employee e, dept d where e.deptid = d.deptid;
#Explain Query Plan
Execute "explain query plan", to get information about the table that is getting used in a query or view. This is very helpful when you are debugging a complex query with multiple joins
on several tables;
sqlite> explain query plan select * from empdept;
#Attach and Detach Database
When you have multiple database, you can use attach command to execute queries across database
For example, if you have two database that has the same table name with different data, you can create a union query across the database to view the combined records
sqlite> attach database 'company1.db' as c1;
sqlite> attach database 'company2.db' as c2;
sqlite> detach c1;
Strip command is used mostly in situations where you want to produce a production quality object file which contains minimum required information so that is can be light weight. You can also use it if you don't want your exectuable or object file to get reverse engineered
SNMP stands for Simple Network Management protocol
This protocol started becoming popular and it forms the basis of network management today
Through snmp one can retrieve information about network devices like routers,printers,hubs or even normal computers. The information that can be retrieved through snmp is endless.
Examples
- System up time
- CPU usage level
- Disk usage level
- Network settings etc
SNMP system design
A computer network system that uses SNMP for network management consists of the three fundamental componenets
1. The SNMP manager: It is a software that usually runs on the machine of network administrator or any human manager managing the computer network
2. The SNMP agent: It is a software that usually runs on the network node that is to be monitored. This node could be a printer. router etc
3. The SNMP MIB: MIB stands for management information base. This component makes sure that the data exchange between the manager and the agent remains structured
SNMP uses UDP
At the transport layer, the protocol used for SNMP message transportation is UDP. This is because UDP outperforms TCP in lossy networks where congestion is usually very high. One thing to remember is to fine tune the time-outs of UDP to fetch the best performance in lossy networks
UNAME
$uname -s (Which displays the kernel name)
Linux
$uname -m (Machine hardware name)
x86_64
$uname -o (Operating System name )
GNU/Linux
Octal Dump
od command in Linux is used to output the contents of a file in different formats with the octal format being the default
Hexadecimal (using -x along with -A)
Octal (using -o along with -A)
Decimal (using -d along with -A)
lsof (Identify Open Files)
lsof stands for List Open Files. It is a command line utility which is used to list the information about the files that are opened by various processes. In unix, everything is a file (pipes,sockets,directories,devices etc). So by using lsof, you can get the information about any opened files.
- List processes which opened a specific file
You can list only the processes which opened a specific file, by providing the filename as arguments
#lsof /var/log/syslog
- List openned files under a directory
You can list the processes which opened files under a specified directory using '+D' option.
#lsof +D /var/log/
- List openned files based on process names starting with
#lsof -c ssh -c init
- List processes using a mount point
#lsof /home
#lsof +D /home/
- List files opened by a specific user
In order to find the list of files opened by a specific users, use '-u' option
#lsof -u gyani
- List all open files by a specific process
#lsof -p 1753
- Kill all process that belongs to a particular user
#kill -9 `lsof -t -u gyani`
- List processes which are listening on a particular port
#lsof -i :25
- List all TCP or UDP connections
#lsof -i tcp; lsof -i udp;
Bash Tweaks
#sed '$aEND-OF-FILE' yourFile # Add a word to everyline of the file at the end of the line
#sed -i 's/^/This /' infile or sed 's/^/This /' infile # Add a word to everyline of the file at the begining of the line
#sed "s/$/,/g" filename #At the end of everyline I need to add a comma at the end of the file
Use sed where ^ is a regular expression, signifying the beginning of the line
