www.carfield.com.hk

filesystem.txt

2023-12-08T04:22:32Z

Advisory Locking and Mandatory Locking - http://www.thegeekstuff.com/2012/04/linux-file-locking-types/

Basic info of inode - http://www.thegeekstuff.com/2012/01/linux-inodes/

Explanation of /proc filesystem - http://www.redhat.com[..]anual/ref-guide/s1-proc-directories.html

Linux file system structure - http://www.thegeekstuff.com/2010/09/linux-file-system-structure/

File system for archive files - http://linuxtoy.org/archives/avfs.html

BeOS file system, other than discuss about BeOS FS, also discussing about evolution of design of file system - http://arstechnica.com[..]rce/news/2010/06/the-beos-filesystem.ars http://arstechnica.com[..]03/past-present-future-file-systems.ars/

ZFS - http://www.thegeekstuff.com/2015/07/zfs-on-linux-zpool/ https://clusterhq.com/2014/09/11/file-systems-data-loss-zfs/ http://www.linuxjournal.com/content/zfs-linux https://arstechnica.com[..]derstanding-zfs-storage-and-performance/

The file size of command "df" return is compressed data, raw disk usage
If we want compress amount, we need to get the compressratio using /usr/sbin/zfs
We can get more system detail via perl package Sun::Solaris::Kstat , we can google "arc_summary.pl" as an example

Using ZFS as version control - http://zef.me/blog/6023/who-needs-git-when-you-got-zfs

When the Linux single-purpose philosophy fails: NFS - https://raduzaharia.medium.com[..]urpose-philosophy-fails-nfs-512ccd21d98c

find.txt

2023-07-30T14:04:42Z

An question about how to remove unused package from MacOSX turn out to be an example of "find" command:

http://forums.osxfaq.com/viewtopic.php?p=57887#57887
http://www.osxfaq.com/tips/unix-tricks/week98/tuesday.ws
http://javarevisited.blogspot.com[..]find-command-in-unix-examples-basic.html

File Search tips in Linux - http://discuss.joelonsoftware.com[..]iscussTopicParent=18272&ixDiscussGroup=3

Another search tip, example of find base on time and size, and how to compare with other files - http://viralpatel.net[..]e-useful-unix-file-finding-commands.html

Like search by name, depth, size - http://www.thegeekstuff.com[..]5-practical-linux-find-command-examples/

use 'find' to delete branch of file: http://keithdevens.com/weblog/archive/2005/Jul/05/find.rm

By time - http://www.thegeekstuff.com[..]unix-linux-find-command-examples-part-2/

Use of empty - http://www.thegeekstuff.com[..]010/03/find-empty-directories-and-files/

Find Anything You Need with fzf - https://medium.com[..]the-linux-fuzzy-finder-tool-f48dfd0092b4

shell.txt

2023-05-19T13:22:08Z

Sum time in script - http://www.markhneedham.com[..]-summing-the-total-time-from-a-log-file/

Special bash parameters - http://javarevisited.blogspot.com[..]ial-bash-parameters-in-script-linux.html

ls sort by file size - http://www.unix.com[..]ns-answers/6518-sorting-ls-filesize.html

How to use "~" - http://www.thegeekstuff.com/2010/06/bash-tilde-expansion

use rlwrap to enable BASH like editing in SQLplus, and use screen to presist terminal sessions - http://binkley.blogspot.com[..]miracle-utilities-rlwrap-and-screen.html

Ctrl+r searches through history
Use the history command plus !# to run old commands
Shell expansion using {} , e.g. cp foo.sh{,.old} == cp foo.sh foo.sh.old
zsh have good auto complete

http://www.pgrs.net/2007/9/6/useful-unix-tricks

mkdir -p creates nested directories
Use less instead of tail
lsof shows open files
ps will show process trees with the f flag

http://www.pgrs.net/2007/10/8/useful-unix-tricks-part-2

!! is the previous command in the shell history
Use vim -b to show nonprintable characters
** is a recursive wildcard in zsh
cd – will return to the previous folder
Use ctrl+z and kill %1 to kill a process that will not die
pwdx shows the working directory of a process
Use sh -x to debug shell scripts
sysctl replaces /proc on macs

http://pgrs.net/2009/2/20/useful-unix-tricks-part-3

ps -eo pcpu,pid,args | sort -n => check CPU%
grep -sq "" /etc/lsb-release && lsb_release -rd => run command if file exist

More about ps and free - http://www.thegeekstuff.com/2010/07/3-shell-scripts/

HTML tools for commandline - http://www.macdevcenter.com/lpt/a/6343

Bash history - http://www.talug.org/events/20030709/cmdline_history.html

Tips of using shell, like saving all history - http://blog.andrewhays.net/love-your-terminal

Standard Shell completion:

Variablename completion
Username completion
Executable completion
Filename and directory completion
Hostname completion

http://www.thegeekstuff.com/2013/11/bash-standard-completion/

replace
cp long_and_complex_file_name long_and_complex_file_name.backup
with this:
cp long_and_complex_file_name{,.backup}

http://www.jacopretorius.net[..]/2013/12/unix-trick-brace-expansion.html

Use the history - http://www.jacopretorius.net/2013/12/my-favorite-unix-trick.html

How to use xargs - http://www.thegeekstuff.com/2013/12/xargs-examples/

Nice presentation of man page - http://explainshell.com/explain?cmd=+ls+-ltr

Parallel shell - http://www.ubuntugeek.com/pssh-parallel-ssh-tools.html

A Beginner’s Guide to tmux - https://medium.com[..]s/a-beginners-guide-to-tmux-7e6daa5c0154

vim.txt

2022-10-21T12:30:48Z

security.txt

2022-07-20T01:36:17Z

Built-in checksum tool "cksum" - https://www.freecodecamp.org[..]inux-how-to-check-if-two-files-are-same/

Disable root password - http://www.debianadmin.com[..]le-and-disable-ubuntu-root-password.html

How can we fine tune permission more than chmod - http://man-wiki.net/index.php/3:cap_from_text

1. From a shell account on a Unix box behind the firewalled network do this:

ssh -N -R 3389:localhost:22 user@remote-unix-box.net

2. Leave work and go home. From a shell account on remote-unix-box.net do this:

ssh 127.0.0.1 -p 3389

http://16systems.com/acls.txt

display.txt

2022-03-03T23:10:30Z

Make your Bash scripts more interactive and futuristic with these tips. - https://levelup.gitconnected.com[..]only-a-few-programmers-know-4abb58ddadad

startup.txt

2022-02-06T14:46:59Z

systemd, An introduction! - https://dev.to/ethanrodrigo/systemd-an-introduction-lc6

Linux.txt

2022-01-19T00:14:25Z

How to get the system thread version - getconf GNU_LIBPTHREAD_VERSION http://groups.google.com[..]wse_thread/thread/81ff4b5d1ea3372d?hl=en

How to prevent module getting load, by edit /etc/modprobe.d/blacklist.conf : http://www.basshero.org/62/how-to-disable-devices-in-ubuntu/

Standard Linux error code - http://www.thegeekstuff.com/2010/10/linux-error-codes/

http://www.thegeekstuff.com[..]11/03/linux-performance-monitoring-intro

Discussion about why Linux is way faster than Windows for huge number of small file processing - http://stackoverflow.com[..]s-to-go-as-fast-as-linux-for-compiling-c http://games.greggman.com/game/why-is-windows-so-slow/

"sync; sudo echo 3 > /proc/sys/vm/drop_caches" - http://www.ubuntugeek.com[..]ow-to-clear-cached-memory-on-ubuntu.html

Funny linux commands - http://www.pixelstech.net/article/1370744983-Funny-Linux-commands

25 years of linux - https://medium.freecodecamp.com[..]acts-about-linux-c8d8ac30076d#.m6b9wnhni

basic-principle-of-Linux-privilege-control - http://www.infoq.com[..]sic-principle-of-Linux-privilege-control

New approach to save memory - https://code.fb.com[..]ng-oomd-a-new-approach-to-handling-ooms/

a-zoological-guide-to-kernel-data-structures - https://medium.com[..]e-to-kernel-data-structures-4c68207e186b

The Arch Linux Handbook - https://www.freecodecamp.org/news/how-to-install-arch-linux/

automation.txt

2021-11-03T12:09:14Z

What is Ansible? A Tool to Automate Parts of Your Job - https://www.freecodecamp.org/news/what-is-ansible

performance.txt

2021-07-10T13:32:48Z

Few config suggestions - http://blog.monitis.com[..]20-linux-server-performance-tips-part-2/

Linux kernel profiling with perf - https://perf.wiki.kernel.org/index.php/Tutorial http://lxr.free-electrons.com/source/tools/perf/design.txt

Intel VTune Amplifier profiler - http://software.intel.com/en-us/intel-vtune-amplifier-xe

Oracle solaris studio - http://www.oracle.com[..]torage/solarisstudio/overview/index.html

http://www.brendangregg.com/linuxperf.html <- "This page links to various Linux performance material I've created, including the tools maps on the right."

Simple config to kernel can get meaningful performance improvement, if needed - https://blog.acolyer.org[..]nce-evolution-of-linuxs-core-operations/

monitoring.txt

2020-12-29T14:03:33Z

Using unhide to check hidden process - http://www.ubuntugeek.com[..]unhide-the-opensource-forensic-tool.html

fuser, check which user using the file or directory - http://www.thegeekstuff.com/2012/02/linux-fuser-command/

htop, look much better than top - http://nerds-central.blogspot.com[..]ght-no-i-am-loving-htop-it-is-linux.html http://www.thegeekstuff.com/2011/09/linux-htop-examples

Tutorial of both lsof and pstack - http://myhowto.org[..]ternals-with-lsof-and-strace/#strace_run

Tutorial of SAR - http://www.thegeekstuff.com/2011/03/sar-examples . In Solaris, you can check SAR from previous days using "sar -f /var/adm/sa/sa30 " ( last 30th )

If we like to see the detail from "ps" command:
solaris: /usr/ucb/ps -awwx , or, use pargs, but need sudo to that user - http://javarevisited.blogspot.com.au[..]ow-long-argument-of-running-process.html
linux: ps -aef --cols [XXXX]

prstat - use prstat instead of ps in solaris http://developers.sun.com/solaris/articles/prstat.html
How to check physical memory on solaris? /usr/sbin/prtconf | grep Memory http://www.dbapool.com/faqs/Q_116.html

Other forensics tools - http://www.forensicfocus.com[..]om/computer-forensics-software-intro.php

Sun monitoring tools collections - http://blogs.sun.com[..]try/solaris_performance_monitoring_tools http://tech.varghees.com[..]sed-system-monitoring-commands-in-linux/

time, basic unix util - http://www.thegeekstuff.com/2013/10/time-command-format

http://www.baptiste-wicht.com[..]ow-level-discovery-cores-cpus-hard-disk/

10 commands for system monitoring - http://techblog.netflix.com[..]1/linux-performance-analysis-in-60s.html http://www.infoq.com/cn/news/2015/12/linux-performance

https://linuxcommando.blogspot.com[..]nxi-swiss-army-knife-for-displaying.html

Top explain - https://dzone.com/articles/different-cpu-times-unixlinux-top

network.txt

2020-10-11T13:30:21Z

TCP tuning - http://rodrickbrown.com/blog/?p=127

UNIX network performance analysis - http://www.ibm.com[..]works/aix/library/au-networkperfanalysis

Netstat tips - http://www.thegeekstuff.com/2010/03/netstat-command-examples/

Introduction to a lot of difference commands - http://tech.varghees.com[..]work-monitoring-commands-in-linux-part-i

Commandline tools which can do most HTTP message - http://www.thekua.com[..]11/scripting-web-applications-with-curl/

Tools to show contents from network - http://coding.smashingmagazine.com[..]/powerful-command-line-tools-developers/

http://java67.blogspot.com.au[..]nd-to-find-ip-address-from-hostname.html

/proc/sys/net/core/
/rmem_default: The default setting of the socket receive buffer in bytes.
/rmem_max: The maximum receive socket buffer size in bytes.
/wmem_default: The default setting (in bytes) of the socket send buffer.
/wmem_max: The maximum send socket buffer size in bytes.
Only for Linux - http://www.lainoox.com/view-or-set-socket-receive-buffer-size/

Getting IP of the host: http://whatismyipaddress.com/ "wget http://ipecho.net/plain -O - -q ; echo" from - http://ru05team.blogspot.com.au[..]ou-want-to-check-you-current-public.html

Use tcpdump and tshark to monitor network activity - http://jvns.ca/blog/2016/03/16/tcpdump-is-amazing/

一篇文章搞定 Nginx 反向代理与负载均衡 - https://xie.infoq.cn/article/d7ad4d2500d6ce6a4ce3b9c16

text.txt

2017-08-06T09:21:16Z

10 awk tips - http://www.catonmat.net/blog/ten-awk-tips-tricks-and-pitfalls/

8 Powerful Awk Built-in Variables – FS, OFS, RS, ORS, NR, NF, FILENAME, FNR - http://www.thegeekstuff.com[..]iables-fs-ofs-rs-ors-nr-nf-filename-fnr/

Tutorial of text editing commands - http://www.ibm.com[..]works/aix/library/au-unixtext/index.html

wc and nl tutorial - http://www.thegeekstuff.com/2013/02/wc-nl-examples/ https://enkipro.com/insight/558ac712249a234c00842b68

tips for uniq, like show only items have duplications - http://www.thegeekstuff.com/2013/05/uniq-command-examples/

Tips of "cat", include display by column and by delimiter - http://www.thegeekstuff.com/2013/06/cut-command-examples/

Using lynx to convert HTML to text - https://enkipro.com/insight/558aaea45800843200bca364

file.txt

2017-08-06T09:20:16Z

Various ways to working with file - http://www.catonmat.net/blog/bash-one-liners-explained-part-one/

Use truncate to pad the file size up - http://www.lainoox.com/truncate-shrink-extend-files/

Making a file unalterable with chattr - https://enkipro.com/insight/558ac712249a234c00842b68

development.txt

2017-06-14T10:34:19Z

A bash debugger - http://bashdb.sourceforge.net/

Really really good tips - http://fahdshariff.blogspot.com.au[..]3/10/shell-scripting-best-practices.html

Style guide - http://www.kfirlavi.com[..]og/2012/11/14/defensive-bash-programming

Script checker - https://www.shellcheck.net/

How to write a shell - https://indradhanush.github.io/blog/writing-a-unix-shell-part-1 https://indradhanush.github.io/blog/writing-a-unix-shell-part-2

desktop.txt

2017-01-20T03:52:35Z

Some useful tools - https://nicolas.perriault.net/code/2016/from-osx-to-ubuntu/

Tuning - http://www.akitaonrails.com[..]1/17/optimizing-linux-for-slow-computers

bug.txt

2016-11-30T04:57:15Z

25 years old bug - http://www.osnews.com/story/19731/The-25-Year-Old-UNIX-Bug

The problem of copying to other machine and screw up the permission - http://unixhorrorstories.blogspot.hk[..]nix-horror-stories-good-thing-about.html

reference.txt

2016-10-06T15:23:20Z

A very nice sectioned list of unix command - http://en.wikipedia.org/wiki/List_of_Unix_programs

Backup a list of directories - http://linuxintegrators.com/acoliver/code/?permalink=x-0272.html

A quick reference - http://www.cs.usfca.edu/~parrt/course/601/lectures/unix.util.html

Tuning cheat sheet - http://www.performancewiki.com/linux-tuning.html

Common and difference between systems - http://www.thegeekstuff.com/2012/01/linux-unix-kernel/

BSD for linux user - http://www.over-yonder.net/~fullermd/rants/bsd4linux/01

Actually a quick tutorial - https://sanctum.geek.nz/arabesque/series/unix-as-ide/

bash.txt

2016-04-01T03:29:26Z

How to use eval, hash, readonly, shift, getopts, set, unset, let and shopt - http://www.thegeekstuff.com/2010/08/bash-shell-builtin-commands

Sample bash completion

SSH_COMPLETE=( $(cut -f1 -d' ' ~/.ssh/known_hosts |\
tr ',' '\n' |\
sort -u |\
grep -e '[:alpha:]') )
complete -o default -W "${SSH_COMPLETE[*]}" ssh

http://kasparov.skife.org/blog/stuff/starting-a-meme.html

Check history statistic - brianm@binky:~$ history | awk {'print $2'} | sort | uniq -c | sort -k1 -rn | head
164 svn
52 cd
42 ssh
32 sudo
22 git
16 ls
16 for
14 echo
13 man
10 curl
brianm@binky:~$

http://bluebones.net/2008/04/history-meme/

easy way to do "[-h|--help][-n|--name ]" - http://binkley.blogspot.com/2016/03/bash-long-options.html

grep.txt

2016-02-03T14:01:03Z

count, highlight and by position - http://www.thegeekstuff.com[..]15-practical-unix-grep-command-examples/

Few examples of how to use grep - http://www.thegeekstuff.com/2011/10/grep-or-and-not-operators http://javarevisited.blogspot.com[..]xamples-of-grep-command-in-unix-and.html

Using regex - http://www.thegeekstuff.com[..]/01/regular-expressions-in-grep-command/ http://www.thegeekstuff.com[..]mand-with-10-examples-%E2%80%93-part-ii/

locate.txt

2016-02-03T13:55:53Z

http://www.thegeekstuff.com/2012/03/locate-command-examples/

Embedded.txt

2015-12-30T07:09:53Z

Embedded Programming with the GNU Toolchain - http://www.bravegnu.org/gnu-eprog/

tracing.txt

2015-11-09T07:34:36Z

Look like a nice tools provide more information for performance issue - http://oprofile.sourceforge.net/examples/

Check CPU problem with prstat
http://javaeesupportpatterns.blogspot.ca[..]stat-solaris-pinpoint-high-cpu-java.html
http://javaeesupportpatterns.blogspot.ca[..]stat-linux-how-to-pinpoint-high-cpu.html
http://javaeesupportpatterns.blogspot.ca[..]prstat-aix-how-to-pinpoint-high-cpu.html

http://www.thegeekstuff.com/2012/03/reverse-engineering-tools

If we like to trace process detail flow, system call and file opened - http://www.ussg.iu.edu/UAU/advcomm/truss.html http://fuse4bsd.creo.hu/localcgi/man-cgi.cgi?truss+1

-l Includes the id of the responsible lightweight process
(LWP) with each line of trace output. If -f is also
specified, both the process-id and the LWP-id are
included.

If the process hang and we like to check why, can try pstack - http://www.google.com[..]=pstack&sourceid=opera&ie=utf-8&oe=utf-8

Check where is the core dump to - http://groups.google.com[..]s/browse_thread/thread/9f46dfb277c72cc8#

Use gcore to generate coredump for running process - http://www.gsp.com/cgi-bin/man.cgi?section=1&topic=gcore

If we like to check the which process id using which port, we can use lsof or "pfiles /proc/*"
1) save the output of "pfiles /proc/*" to a file
2) grep the file for that port, then find the PID, e.g., say here is the part of the output:


5859:   java -Djava.awt.headless=true -DX_R_BRGD -Xms512M -Xmx1024M -Djava.io.
  Current rlimit: 1024 file descriptors
   0: S_IFCHR mode:0666 dev:85,0 ino:980953 uid:0 gid:3 rdev:13,2
      O_RDONLY|O_LARGEFILE

....... ( 125 FD after) ......

 125: S_IFSOCK mode:0666 dev:313,0 ino:40404 uid:0 gid:0 size:0
      O_RDWR
        sockname: AF_INET 10.152.22.22  port: 18075
        peername: AF_INET 169.19.21.63  port: 41792

Then PID 5859 is the process working with the port 18075, however, it only work for process which is viewable according to your permission setting

Another probably easier way, and provide more information, is using

lsof -i :[port]

- http://javarevisited.blogspot.com[..]-unix-netstat-lsof-command-examples.html

If we like to check the which process id using which file, we can use fuser - http://linuxcommand.org/man_pages/fuser1.html

Check host and port that any process accessing using lsof - http://www.geekology.co.za[..]-are-accessing-internet-on-unix-machine/

How do I monitor or view the thread count of a certain process - ps -o pid,comm,user,thcount -p - http://superuser.com[..]thread-count-of-a-certain-process-on-aix

Examples of using strace, use -e to trace function call is useful - http://www.thegeekstuff.com/2011/11/strace-examples

The nm commands provides information on the symbols being used in an object file or executable file. - http://www.thegeekstuff.com/2012/03/linux-nm-command/

Use pmap -s -x to look at what's in the virtual address space of yr process and what portion thereof is resident in physical memory.

sed.txt

2015-08-21T02:01:52Z

http://www.catonmat.net/blog/worlds-best-introduction-to-sed/

http://javarevisited.blogspot.com.au[..]-command-examples-in-unix-and-linux.html

http://www.markhneedham.com[..]5/08/13/sed-using-environment-variables/

time.txt

2015-08-20T15:40:08Z

http://juliusdavies.ca/posix_clocks/clock_realtime_linux_faq.html http://efreedom.com/Question/1-6814792/Clock-Gettime-Erratic

Unix timezone - https://www.vmware.com[..]isdk400pubs/ReferenceGuide/timezone.html