前言
linux和windows相比,很多管控都需要用命令來操作。windows以直觀的可視化的方式操作,特別適合在桌面端PC上操作執行相應的軟件。
而Linux命令行方式的操作,特別是在服務器端編程、管理、運維方面,更加簡單、短小、精悍。短短一行組合命令,即可完成在windows需要各種加工、整合的復雜高效的功能操作。
1.進程管理
w 顯示當前在線用戶情況
my_adm?pts/0????111.111.111.111????三15???24:58???2:51???0.02s?sshd:?my_admin?[priv]root?????pts/3????111.111.111.112??????13:15????3:47m??0.35s??0.35s?-bashmy_adm?pts/5????111.111.111.113????15:14????2.00s??0.54s??0.02s?sshd:?my_admin?[priv]
pkill -kill -t pts/????殺掉指定名字的進程,如上述的pts/5
2.系統信息
arch?顯示機器的處理器架構(1)?uname?-m?顯示機器的處理器架構(2)?uname?-r?顯示正在使用的內核版本?dmidecode?-q?顯示硬件系統部件?-?(SMBIOS?/?DMI)?hdparm?-i?/dev/hda?羅列一個磁盤的架構特性?hdparm?-tT?/dev/sda?在磁盤上執行測試性讀取操作?cat?/proc/cpuinfo?顯示CPU?info的信息?cat?/proc/interrupts?顯示中斷?cat?/proc/meminfo?校驗內存使用?cat?/proc/swaps?顯示哪些swap被使用?cat?/proc/version?顯示內核的版本?cat?/proc/net/dev?顯示網絡適配器及統計?cat?/proc/mounts?顯示已加載的文件系統?lspci?-tv?羅列?PCI?設備?lsusb?-tv?顯示?USB?設備?date?顯示系統日期?ctime=`date?+%Y-%m-%dT%k:%M:%S`?#格式化時間,如2018-01-13T11:09:19,?注意%k?與?%H的區別,前者返回9,后者返回09.YESTERDAY=`date?+%Y-%m-%d?-d?"-1?days"`?獲取昨日日期cal?2007?顯示2007年的日歷表?date?041217002007.00?設置日期和時間?-?月日時分年.秒?clock?-w?將時間修改保存到?BIOS
查詢網關地址(如將centos服務器網絡獲取類型由dhcp切換為static時,需要獲取到ip及網關信息進行固化配置時會用到,如本地ip為10.34.0.123,則通過如下命令查詢到網關為第一個3.254):
[root@file-server?~]#?netstat?-rnKernel?IP?routing?tableDestination?????Gateway?????????Genmask?????????Flags???MSS?Window??irtt?Iface0.0.0.0?????????10.34.3.254?????0.0.0.0?????????UG????????0?0??????????0?enp3g010.34.0.0???????0.0.0.0?????????255.255.252.0???U?????????0?0??????????0?enp3g0192.168.1.0?????0.0.0.0?????????255.255.255.0???U?????????0?0??????????0?virbr0
CentOS / RedHat 系列,查看操作系統信息(如果沒有lsb_release命令, 使用”yum install redhat-lsb”安裝):
[root@server-test?online]#?lsb_release?-aLSB?Version:?:core-4.1-amd64:core-4.1-noarchDistributor?ID:?CentOSDescription:?CentOS?Linux?release?7.5.1804?(Core)?Release:?7.5.1804Codename:?Core
3.關機 (系統的關機、重啟以及登出 )
shutdown?-h?now?關閉系統(1)?init?0?關閉系統(2)?telinit?0?關閉系統(3)?shutdown?-h?hours:minutes?&?按預定時間關閉系統?shutdown?-c?取消按預定時間關閉系統?shutdown?-r?now?重啟(1)?reboot?重啟(2)?logout?注銷
4.文件和目錄
cd?/home?進入?'/?home'?目錄'?cd?..?返回上一級目錄?cd?../..?返回上兩級目錄?cd?進入個人的主目錄?cd?~user1?進入個人的主目錄?cd?-?返回上次所在的目錄?pwd?顯示工作路徑?ls?查看目錄中的文件?ls?-F?查看目錄中的文件?ls?-l?顯示文件和目錄的詳細資料?ls?-a?顯示隱藏文件?ls?*[0-9]*?顯示包含數字的文件名和目錄名?tree?顯示文件和目錄由根目錄開始的樹形結構(1)?lstree?顯示文件和目錄由根目錄開始的樹形結構(2)?mkdir?dir1?創建一個叫做?'dir1'?的目錄'?mkdir?dir1?dir2?同時創建兩個目錄?mkdir?-p?/tmp/dir1/dir2?創建一個目錄樹?rm?-f?file1?刪除一個叫做?'file1'?的文件'?rmdir?dir1?刪除一個叫做?'dir1'?的目錄'?rm?-rf?dir1?刪除一個叫做?'dir1'?的目錄并同時刪除其內容?rm?-rf?dir1?dir2?同時刪除兩個目錄及它們的內容?mv?dir1?new_dir?重命名/移動?一個目錄?cp?file1?file2?復制一個文件?cp?dir/*?.?復制一個目錄下的所有文件到當前工作目錄?cp?-a?/tmp/dir1?.?復制一個目錄到當前工作目錄?cp?-a?dir1?dir2?復制一個目錄?ln?-s?file1?lnk1?創建一個指向文件或目錄的軟鏈接?ln?file1?lnk1?創建一個指向文件或目錄的物理鏈接?touch?-t?0712250000?file1?修改一個文件或目錄的時間戳?-?(YYMMDDhhmm)?file?file1?outputs?the?mime?type?of?the?file?as?text?iconv?-l?列出已知的編碼?iconv?-f?fromEncoding?-t?toEncoding?inputFile?>?outputFile?creates?a?new?from?the?given?input?file?by?assuming?it?is?encoded?in?fromEncoding?and?converting?it?to?toEncoding.?find?.?-maxdepth?1?-name?*.jpg?-print?-exec?convert?"{}"?-resize?80x60?"thumbs/{}"?;?batch?resize?files?in?the?current?directory?and?send?them?to?a?thumbnails?directory?(requires?convert?from?Imagemagick)
5.文件搜索
find?/?-name?file1?從?'/'?開始進入根文件系統搜索文件和目錄?find?/?-user?user1?搜索屬于用戶?'user1'?的文件和目錄?find?/home/user1?-name?*.bin?在目錄?'/?home/user1'?中搜索帶有'.bin'?結尾的文件?find?/usr/bin?-type?f?-atime?+100?搜索在過去100天內未被使用過的執行文件?find?/usr/bin?-type?f?-mtime?-10?搜索在10天內被創建或者修改過的文件?find?/?-name?*.rpm?-exec?chmod?755?'{}'?;?搜索以?'.rpm'?結尾的文件并定義其權限?^????????????????????????find?/tmp?-name?*.hprof?-exec?rm?-f?{}?;批量刪除java堆棧.hprof文件find?/?-xdev?-name?*.rpm?搜索以?'.rpm'?結尾的文件,忽略光驅、捷盤等可移動設備?locate?*.ps?尋找以?'.ps'?結尾的文件?-?先運行?'updatedb'?命令?whereis?halt?顯示一個二進制文件、源碼或man的位置?which?halt?顯示一個二進制文件或可執行文件的完整路徑?grep?-rn?"query_string"?*??Linux目錄下全局查找所有文件中是否包含指定字符串(-r:遞歸;-n:顯示行號)
6.掛載一個文件系統
mount?/dev/hda2?/mnt/hda2?掛載一個叫做hda2的盤?-?確定目錄?'/?mnt/hda2'?已經存在?umount?/dev/hda2?卸載一個叫做hda2的盤?-?先從掛載點?'/?mnt/hda2'?退出?fuser?-km?/mnt/hda2?當設備繁忙時強制卸載?umount?-n?/mnt/hda2?運行卸載操作而不寫入?/etc/mtab?文件-?當文件為只讀或當磁盤寫滿時非常有用?mount?/dev/fd0?/mnt/floppy?掛載一個軟盤?mount?/dev/cdrom?/mnt/cdrom?掛載一個cdrom或dvdrom?mount?/dev/hdc?/mnt/cdrecorder?掛載一個cdrw或dvdrom?mount?/dev/hdb?/mnt/cdrecorder?掛載一個cdrw或dvdrom?mount?-o?loop?file.iso?/mnt/cdrom?掛載一個文件或ISO鏡像文件?mount?-t?vfat?/dev/hda5?/mnt/hda5?掛載一個Windows?FAT32文件系統?mount?/dev/sda1?/mnt/usbdisk?掛載一個usb?捷盤或閃存設備?mount?-t?smbfs?-o?username=user,password=pass?//WinClient/share?/mnt/share?掛載一個windows網絡共享
7.磁盤空間
df?-h?顯示已經掛載的分區列表?ls?-lSr?|more?以尺寸大小排列文件和目錄?du?-sh?dir1?估算目錄?'dir1'?已經使用的磁盤空間'?du?-sk?*?|?sort?-rn?以容量大小為依據依次顯示文件和目錄的大小?rpm?-q?-a?--qf?'%10{SIZE}t%{NAME}n'?|?sort?-k1,1n?以大小為依據依次顯示已安裝的rpm包所使用的空間?(fedora,?redhat類系統)?dpkg-query?-W?-f='${Installed-Size;10}t${Package}n'?|?sort?-k1,1n?以大小為依據顯示已安裝的deb包所使用的空間?(ubuntu,?debian類系統)?hdfs?dfs?-du?/dw/default?|?sort?-rn?|?head?-n?10?|?awk?'{printf("%.2ftt%.2ftt%stn",$1/1024/1024/1024,"t"$2/1024/1024/1024,"t"$3)}'??查詢hdfs文件系統中表文件大小,按從大到小的順序排列(取前10列),單位GB
8.系統負載 — top
top?-d?20?-p?1303?????????????????????????????將進程號1303的系統負載,每隔20秒刷新一次。英文狀態下,按住c鍵,將展示進行的詳細環境信息,對于java程序調試來說,非常友好。top?-d?20?-n?3?-b?>?test.txt??????????????????每隔20秒,一共執行3次,?將統計結果導入到test.txt文件中。
top命令顯示不全,添加-w參數:
命令為:top -b -n 1
-
-b為 批處理模式, -
-n為刷新的次數
發現信息顯示不全,最后man top,加一個參數w后,完全顯示
top -b -n 1 -w 512
如果需要顯示完整的COMMAND命令,使用top -c參數
top -c -bw 500
查看完整進程名, 按500個字符長度查看(這樣基本可以查看到完整的命令) 其他參數列表,官方解釋:
1.?COMMAND-LINE?Options???????The?command-line?syntax?for?top?consists?of:??????????-hv|-bcHiOSs?-d?secs?-n?max?-u|U?user?-p?pid?-o?fld?-w?[cols]????????The?typically?mandatory?switch?('-')?and?even?whitespace?are?completely?optional.????????-h?|?-v??:Help/Version????????????Show?library?version?and?the?usage?prompt,?then?quit.????????-b??:Batch-mode?operation????????????Starts?top?in?Batch?mode,?which?could?be?useful?for?sending?output?from?top?to?other?programs?or?to?a?file.??In?this?mode,?top?will?not?accept?input?and?runs?until?the?iterations?limit?you've?set?with?the?`-n'?command-line?option?or?until?killed.????????-c??:Command-line/Program-name?toggle????????????Starts?top?with?the?last?remembered?`c'?state?reversed.??Thus,?if?top?was?displaying?command?lines,?now?that?field?will?show?program?names,?and?visa?versa.??See?the?`c'?interactive?command?for?additional?information.????????-d??:Delay-time?interval?as:??-d?ss.t?(secs.tenths)????????????Specifies?the?delay?between?screen?updates,?and?overrides?the?corresponding?value?in?one's?personal?configuration?file?or?the?startup?default.??Later?this?can?be?changed?with?the?`d'?or?`s'?interactive?commands.????????????Fractional??seconds??are?honored,?but?a?negative?number?is?not?allowed.??In?all?cases,?however,?such?changes?are?prohibited?if?top?is?running?in?Secure?mode,?except?for?root?(unless?the?`s'?command-line?option?was?used).??For?additional?information?on????????????Secure?mode?see?topic?6a.?SYSTEM?Configuration?File.????????-H??:Threads-mode?operation????????????Instructs?top?to?display?individual?threads.??Without?this?command-line?option?a?summation?of?all?threads?in?each?process?is?shown.??Later?this?can?be?changed?with?the?`H'?interactive?command.???????-i??:Idle-process?toggle????????????Starts?top?with?the?last?remembered?`i'?state?reversed.??When?this?toggle?is?Off,?tasks?that?have?not?used?any?CPU?since?the?last?update?will?not?be?displayed.??For?additional?information?regarding?this?toggle?see?topic?4c.?TASK?AREA?Commands,?SIZE.????????-n??:Number-of-iterations?limit?as:??-n?number????????????Specifies?the?maximum?number?of?iterations,?or?frames,?top?should?produce?before?ending.????????-o??:Override-sort-field?as:??-o?fieldname????????????Specifies?the?name?of?the?field?on?which?tasks?will?be?sorted,?independent?of?what?is?reflected?in?the?configuration?file.??You?can?prepend?a?`+'?or?`-'?to?the?field?name?to?also?override?the?sort?direction.??A?leading?`+'?will?force?sorting??high??to????????????low,?whereas?a?`-'?will?ensure?a?low?to?high?ordering.?????????????This?option?exists?primarily?to?support?automated/scripted?batch?mode?operation.????????-O??:Output-field-names????????????This?option?acts?as?a?form?of?help?for?the?above?-o?option.??It?will?cause?top?to?print?each?of?the?available?field?names?on?a?separate?line,?then?quit.??Such?names?are?subject?to?nls?translation.????????-p??:Monitor-PIDs?mode?as:??-pN1?-pN2?...??or??-pN1,N2,N3?...????????????Monitor?only?processes?with?specified?process?IDs.??This?option?can?be?given?up?to?20?times,?or?you?can?provide?a?comma?delimited?list?with?up?to?20?pids.??Co-mingling?both?approaches?is?permitted.?????????????A?pid?value?of?zero?will?be?treated?as?the?process?id?of?the?top?program?itself?once?it?is?running.?????????????This?is?a?command-line?option?only?and?should?you?wish?to?return?to?normal?operation,?it?is?not?necessary?to?quit?and?restart?top??--??just?issue?any?of?these?interactive?commands:?`=',?`u'?or?`U'.????????????The?`p',?`u'?and?`U'?command-line?options?are?mutually?exclusive.????????-s??:Secure-mode?operation????????????Starts?top?with?secure?mode?forced,?even?for?root.??This?mode?is?far?better?controlled?through?the?system?configuration?file?(see?topic?6.?FILES).????????-S??:Cumulative-time?toggle????????????Starts??top??with??the??last??remembered?`S'?state?reversed.??When?Cumulative?time?mode?is?On,?each?process?is?listed?with?the?cpu?time?that?it?and?its?dead?children?have?used.??See?the?`S'?interactive?command?for?additional?information?regarding?this????????????mode.????????-u?|?-U??:User-filter-mode?as:??-u?|?-U?number?or?name????????????Display?only?processes?with?a?user?id?or?user?name?matching?that?given.??The?`-u'?option?matches?on??effective?user?whereas?the?`-U'?option?matches?on?any?user?(real,?effective,?saved,?or?filesystem).?????????????Prepending?an?exclamation?point?('!')?to?the?user?id?or?name?instructs?top?to?display?only?processes?with?users?not?matching?the?one?provided.?????????????The?`p',?`u'?and?`U'?command-line?options?are?mutually?exclusive.???????-w??:Output-width-override?as:??-w?[?number?]????????????In?Batch?mode,?when?used?without?an?argument?top?will?format?output?using?the?COLUMNS=?and?LINES=?environment?variables,?if?set.??Otherwise,?width?will?be?fixed?at?the?maximum?512?columns.??With?an?argument,?output?width?can?be?decreased?or??increased????????????(up?to?512)?but?the?number?of?rows?is?considered?unlimited.????????????In?normal?display?mode,?when?used?without?an?argument?top?will?attempt?to?format?output?using?the?COLUMNS=?and?LINES=?environment?variables,?if?set.??With?an?argument,?output?width?can?only?be?decreased,?not?increased.??Whether?using?environment?vari‐????????????ables?or?an?argument?with?-w,?when?not?in?Batch?mode?actual?terminal?dimensions?can?never?be?exceeded.????????????Note:?Without?the?use?of?this?command-line?option,?output?width?is?always?based?on?the?terminal?at?which?top?was?invoked?whether?or?not?in?Batch?mode.
9.用戶和群組
groupadd?group_name?創建一個新用戶組?groupdel?group_name?刪除一個用戶組?groupmod?-n?new_group_name?old_group_name?重命名一個用戶組?useradd?-c?"Name?Surname?"?-g?admin?-d?/home/user1?-s?/bin/bash?user1?創建一個屬于?"admin"?用戶組的用戶?useradd?user1?創建一個新用戶?userdel?-r?user1?刪除一個用戶?(?'-r'?排除主目錄)?usermod?-c?"User?FTP"?-g?system?-d?/ftp/user1?-s?/bin/nologin?user1?修改用戶屬性?passwd?修改口令?passwd?user1?修改一個用戶的口令?(只允許root執行)?chage?-E?2005-12-31?user1?設置用戶口令的失效期限?pwck?檢查?'/etc/passwd'?的文件格式和語法修正以及存在的用戶?grpck?檢查?'/etc/passwd'?的文件格式和語法修正以及存在的群組?newgrp?group_name?登陸進一個新的群組以改變新創建文件的預設群組
案例 1:
1-創建用戶、2-追加附加組、3-刪除附加組
useradd?san_zhang?-mpasswd?san_zhang?san_zhang@163.com
從管道輸出中修改密碼(如果非root且可以轉為root,請使用sudo)
echo?san_zhang@163.com?|?sudo?passwd?--stdin?san_zhang
2 將san_zhang追加到附加組hdfs中
可以添加多個附加組,多個組之間用空格隔開,如”hdfs,yarn,spark”),其默認在san_zhang中
usermod?-aG?hdfs?san_zhang
此時查看其所在的用戶組,可以發現其所屬組為san_zhang,附加組為hdfs,如下:
id?san_zhanguid=1001(san_zhang)?gid=1001(san_zhang)?groups=1001(san_zhang),992(hdfs)
引申:如果上一步順序寫錯,誤將hdfs追加到san_zhang用戶組中,如使用了如下錯誤的命令:
usermod?-aG?san_zhang?hdfs
則此時查看hdfs所屬組,就會發現其也在san_zhang組下了。另外,搜索公眾號Linux就該這樣學后臺回復“Linux”,獲取一份驚喜禮包。
id?hdfsuid=995(hdfs)?gid=992(hdfs)?groups=992(hdfs),994(hadoop),1001(san_zhang)
這時再查看san_zhang組,就會看到誤加入的hdfs用戶了
groups?san_zhangsan_zhang?:?san_zhang?hdfs
3 從san_zhang組中刪除誤加入的hdfs用戶
gpasswd?-d?hdfs?san_zhang
執行后的日志:Removing user hdfs from group san_zhang
案例 2:
統計某個分組下有哪些用戶
[root@cdh01 ~]# grep hdfs /etc/group hdfs:x:993:
[root@cdh01 ~]# awk -F : ‘{print?4}’ /etc/passwd | grep ‘993’ hdfs 993
10.文件的權限 – 使用 “+” 設置權限,使用 “-” 用于取消
ls?-lh?顯示權限?ls?/tmp?|?pr?-T5?-W$COLUMNS?將終端劃分成5欄顯示?chmod?ugo+rwx?directory1?設置目錄的所有人(u)、群組(g)以及其他人(o)以讀(r?)、寫(w)和執行(x)的權限?chmod?go-rwx?directory1?刪除群組(g)與其他人(o)對目錄的讀寫執行權限?chown?user1?file1?改變一個文件的所有人屬性?chown?-R?user1?directory1?改變一個目錄的所有人屬性并同時改變改目錄下所有文件的屬性?chgrp?group1?file1?改變文件的群組?chown?user1:group1?file1?改變一個文件的所有人和群組屬性?find?/?-perm?-u+s?羅列一個系統中所有使用了SUID控制的文件?chmod?u+s?/bin/file1?設置一個二進制文件的?SUID?位?-?運行該文件的用戶也被賦予和所有者同樣的權限?chmod?u-s?/bin/file1?禁用一個二進制文件的?SUID位?chmod?g+s?/home/public?設置一個目錄的SGID?位?-?類似SUID?,不過這是針對目錄的?chmod?g-s?/home/public?禁用一個目錄的?SGID?位?chmod?o+t?/home/public?設置一個文件的?STIKY?位?-?只允許合法所有人刪除文件?chmod?o-t?/home/public?禁用一個目錄的?STIKY?位
11.文件的特殊屬性 – 使用 “+” 設置權限,使用 “-” 用于取消
chattr?+a?file1?只允許以追加方式讀寫文件?chattr?+c?file1?允許這個文件能被內核自動壓縮/解壓?chattr?+d?file1?在進行文件系統備份時,dump程序將忽略這個文件?chattr?+i?file1?設置成不可變的文件,不能被刪除、修改、重命名或者鏈接?chattr?+s?file1?允許一個文件被安全地刪除?chattr?+S?file1?一旦應用程序對這個文件執行了寫操作,使系統立刻把修改的結果寫到磁盤?chattr?+u?file1?若文件被刪除,系統會允許你在以后恢復這個被刪除的文件?lsattr?顯示特殊的屬性
12.打包和壓縮文件
bunzip2?file1.bz2?解壓一個叫做?'file1.bz2'的文件?bzip2?file1?壓縮一個叫做?'file1'?的文件?gunzip?file1.gz?解壓一個叫做?'file1.gz'的文件?gzip?file1?壓縮一個叫做?'file1'的文件?gzip?-9?file1?最大程度壓縮?rar?a?file1.rar?test_file?創建一個叫做?'file1.rar'?的包?rar?a?file1.rar?file1?file2?dir1?同時壓縮?'file1',?'file2'?以及目錄?'dir1'?unrar?x?file1.rar?解壓rar包???#如果無unrar命令,參考:Linux?CentOS?7.0?下?rar?unrar的安裝tar?-cvf?archive.tar?file1?創建一個非壓縮的?tarball?tar?-cvf?archive.tar?file1?file2?dir1?創建一個包含了?'file1',?'file2'?以及?'dir1'的檔案文件?tar?-tf?archive.tar?顯示一個包中的內容?tar?-xvf?archive.tar?釋放一個包?tar?-xvf?archive.tar?-C?/tmp?將壓縮包釋放到?/tmp目錄下?tar?-cvfj?archive.tar.bz2?dir1?創建一個bzip2格式的壓縮包?tar?-jxvf?archive.tar.bz2?解壓一個bzip2格式的壓縮包?tar?-cvfz?archive.tar.gz?dir1?創建一個gzip格式的壓縮包?tar?-zxvf?archive.tar.gz?解壓一個gzip格式的壓縮包?zip?file1.zip?file1?創建一個zip格式的壓縮包?zip?-r?file1.zip?file1?file2?dir1?將幾個文件和目錄同時壓縮成一個zip格式的壓縮包?unzip?file1.zip?解壓一個zip格式壓縮包
13.OS包管理器
①、RPM 包 – (Fedora, Redhat及類似系統)
rpm?-ivh?package.rpm?安裝一個rpm包?rpm?-ivh?--nodeps?package.rpm?安裝一個rpm包而忽略依賴關系警告?rpm?-U?package.rpm?更新一個rpm包但不改變其配置文件?rpm?-F?package.rpm?更新一個確定已經安裝的rpm包?rpm?-e?[--nodeps]?package_name.rpm?刪除一個rpm包[--nodeps表示忽略依賴關系]?rpm?-e?--nodeps?`rpm?-qa?|?grep?clickhouse`?批量刪除所有?ClickHouse?已安裝的?rpm?包,并忽略彼此間的依賴關系rpm?-qa?顯示系統中所有已經安裝的rpm包?rpm?-qa?|?grep?httpd?顯示所有名稱中包含?"httpd"?字樣的rpm包?rpm?-qi?package_name?獲取一個已安裝包的特殊信息?rpm?-qg?"System?Environment/Daemons"?顯示一個組件的rpm包?rpm?-ql?package_name?顯示一個已經安裝的rpm包提供的文件列表?rpm?-qc?package_name?顯示一個已經安裝的rpm包提供的配置文件列表?rpm?-q?package_name?--whatrequires?顯示與一個rpm包存在依賴關系的列表?rpm?-q?package_name?--whatprovides?顯示一個rpm包所占的體積?rpm?-q?package_name?--scripts?顯示在安裝/刪除期間所執行的腳本l?rpm?-q?package_name?--changelog?顯示一個rpm包的修改歷史?rpm?-qf?/etc/httpd/conf/httpd.conf?確認所給的文件由哪個rpm包所提供?rpm?-qp?package.rpm?-l?顯示由一個尚未安裝的rpm包提供的文件列表?rpm?--import?/media/cdrom/RPM-GPG-KEY?導入公鑰數字證書?rpm?--checksig?package.rpm?確認一個rpm包的完整性?rpm?-qa?gpg-pubkey?確認已安裝的所有rpm包的完整性?rpm?-V?package_name?檢查文件尺寸、?許可、類型、所有者、群組、MD5檢查以及最后修改時間?rpm?-Va?檢查系統中所有已安裝的rpm包-?小心使用?rpm?-Vp?package.rpm?確認一個rpm包還未安裝?rpm2cpio?package.rpm?|?cpio?--extract?--make-directories?*bin*?從一個rpm包運行可執行文件?rpm?-ivh?/usr/src/redhat/RPMS/`arch`/package.rpm?從一個rpm源碼安裝一個構建好的包?rpmbuild?--rebuild?package_name.src.rpm?從一個rpm源碼構建一個?rpm?包
②、YUM 軟件包升級器 – (Fedora, RedHat及類似系統)
yum?install?package_name?下載并安裝一個rpm包?(如:?yum?-y?install?zip?unzip)yum?localinstall?package_name.rpm?將安裝一個rpm包,使用你自己的軟件倉庫為你解決所有依賴關系?yum?update?package_name.rpm?更新當前系統中所有安裝的rpm包?yum?update?package_name?更新一個rpm包?yum?remove?package_name?刪除一個rpm包?yum?list?|?grep?tmux??列出所有可安裝的軟件清單命令yum?search?package_name?在rpm倉庫中搜尋軟件包?yum?clean?packages?清理rpm緩存刪除下載的包?yum?clean?headers?刪除所有頭文件?yum?clean?all?刪除所有緩存的包和頭文件
③、DEB 包 (Debian, Ubuntu 以及類似系統)
dpkg?-i?package.deb?安裝/更新一個?deb?包?dpkg?-r?package_name?從系統刪除一個?deb?包?dpkg?-l?顯示系統中所有已經安裝的?deb?包?dpkg?-l?|?grep?httpd?顯示所有名稱中包含?"httpd"?字樣的deb包?dpkg?-s?package_name?獲得已經安裝在系統中一個特殊包的信息?dpkg?-L?package_name?顯示系統中已經安裝的一個deb包所提供的文件列表?dpkg?--contents?package.deb?顯示尚未安裝的一個包所提供的文件列表?dpkg?-S?/bin/ping?確認所給的文件由哪個deb包提供
④、APT 軟件工具 (Debian, Ubuntu 以及類似系統)
apt-get?install?package_name?安裝/更新一個?deb?包?apt-cdrom?install?package_name?從光盤安裝/更新一個?deb?包?apt-get?update?升級列表中的軟件包?apt-get?upgrade?升級所有已安裝的軟件?apt-get?remove?package_name?從系統刪除一個deb包?apt-get?check?確認依賴的軟件倉庫正確?apt-get?clean?從下載的軟件包中清理緩存?apt-cache?search?searched-package?返回包含所要搜索字符串的軟件包名稱
14.文本查看、處理
①、查看文件內容
cat?file1?從第一個字節開始正向查看文件的內容?tac?file1?從最后一行開始反向查看一個文件的內容?more?file1?查看一個長文件的內容?less?file1?類似于?'more'?命令,但是它允許在文件中和正向操作一樣的反向操作?head?-2?file1?查看一個文件的前兩行?tail?-2?file1?查看一個文件的最后兩行?tail?-f?/var/log/messages?實時查看被添加到一個文件中的內容
②、文本處理
cat?file1?file2?...?|?command??file1_in.txt_or_file1_out.txt?general?syntax?for?text?manipulation?using?PIPE,?STDIN?and?STDOUT?cat?file1?|?command(?sed,?grep,?awk,?grep,?etc...)?>?result.txt?合并一個文件的詳細說明文本,并將簡介寫入一個新文件中?cat?file1?|?command(?sed,?grep,?awk,?grep,?etc...)?>>?result.txt?合并一個文件的詳細說明文本,并將簡介寫入一個已有的文件中?grep?Aug?/var/log/messages?在文件?'/var/log/messages'中查找關鍵詞"Aug"?grep?^Aug?/var/log/messages?在文件?'/var/log/messages'中查找以"Aug"開始的詞匯?grep?[0-9]?/var/log/messages?選擇?'/var/log/messages'?文件中所有包含數字的行?grep?Aug?-R?/var/log/*?在目錄?'/var/log'?及隨后的目錄中搜索字符串"Aug"?grep?-E?'string_1|string_2'?/var/log/test.log?在日志中同時查詢包含?string_1或包含string_2的記錄#?匹配redis中,?field?中包含以1到4位數字開頭,以"_已下單"結尾的字符,?并將其?value?值帶出來(-A1)redis-cli?-h?111.111.111.111?-p?7001?-c?-a?${PASSWORD}?--raw?hgetall?2022-03-24_ORDER_STAT?|?grep?-E?"^[[:digit:]]{1,4}_已下單"?-A1grep?-Hnr?docker?*?在本層或子文件夾里遞歸查找包含指定字符“docker”的文件,并顯示文件名+行號,如果精確匹配文本,加o參數。參數含義:(H:每次匹配時將文件名打印出來;n:打印字符串所在的行號;r:遞歸查找;o:精確匹配到單詞,多一個字符都不行)
“grep顯示匹配行的上下行內容”專題:
grep?-A2?2?test.txt??返回匹配行和匹配行的后兩行?grep?-B2?2?test.txt??返回匹配行和匹配行的前兩行?grep?-C2?2?text.txt?或者?grep?-A2?-B2?2?test.txt?返回匹配行和匹配行的前后兩行
-
-A -B -C 后面跟阿拉伯數字 -
-A 返回匹配后和它后面的n行,(After,之后)。 -
-B 返回匹配行和它前面的n行,(Before,之前)。 -
-C 返回匹配行和它前后各n行,通-A和-B聯用,(Context,上下文即前后)。
牛逼??!接私活必備的?N?個開源項目!趕快收藏
sed?-i?'s/string1/string2/g'?example.txt?將example.txt文件中的?"string1"?替換成?"string2"(如果不加-i參數,則在內存中替換;?添加-i參數,則在文件中直接替換)sed?'/^$/d'?example.txt?從example.txt文件中刪除所有空白行?sed?'/?*#/d;?/^$/d'?example.txt?從example.txt文件中刪除所有注釋和空白行?echo?'hElloWorld'?|?tr?'[:lower:]'?'[:upper:]'?找出所有的小寫字母并全部替換為大寫字母?sed?-e?'1d'?result.txt?從文件example.txt?中排除第一行?sed?-n?'/string1/p'?查看只包含詞匯?"string1"的行?sed?-e?'s/?*$//'?example.txt?刪除每一行最后的空白字符?sed?-e?'s/string1//g'?example.txt?從文檔中只刪除詞匯?"string1"?并保留剩余全部?sed?-n?'1,5p;5q'?example.txt?查看從第一行到第5行內容?sed?-n?'5p;5q'?example.txt?查看第5行?sed?-e?'s/00*/0/g'?example.txt?用單個零替換多個零sed?-i?'3i?helloword'?test.txt???在test.txt文件的第三行插入‘helloword’字符串
“sed按指定字符串刪除”專題:
用sed刪除匹配到字符串的行:
語法:sed-i’/關鍵字符/d’文件名
舉例1:匹配”etcinstall.sh”
sed?-i?'//etc/install.sh/d'?1.txt
-
-i 表示操作在源文件上生效.否則操作內存中數據,并不寫入文件中. 在分號內的/d表示刪除匹配的行。
舉例2:刪除以a開頭的行
sed?-i?'/^a.*/d'?tmp.txt
-
^a表示開頭是a, .*表示后跟任意字符串
Example-1: ?使用sed進行文本批量替換
#!/bin/bash?ddl_file_path=/c/Users/user/Downloads/sqoop_data/hive_2_mysql??ls?$ddl_file_path?|?while?read?f;doecho?"--------->"$ddl_file_path/$f?##?Template#sed?-i?'s///g'?$ddl_file_path/$f#sed?-i?"s///g"?$ddl_file_path/$f?sed?-i?'s/111.111.111.111:3306/222.222.222.222:3307/g'?$ddl_file_path/$fsed?-i?"s/'password_1'/'password_2'/g"?$ddl_file_path/$f?done?#?echo?-e?"=========>nt?Files?contains?keyword?"/WARN:"?in?path?$ddl_file_path?are?deleted!!!"
#!/bin/bash?set?-x?##?Template#sed?-i?'s///g'?$ddl_file_path/$f#sed?-i?"s///g"?$ddl_file_path/$f?##?Func?1?-?實現在所有文件的第四行、第五行插入指定文本?ddl_file_path=/e/迅雷下載/user_data_after?ls?$ddl_file_path?|?while?read?f;do????????#?如果文件不是文件夾類型,才允許插入操作????if?[?!?-d?$f?];then????????echo?"--------->"$ddl_file_path/$f?????????sed?-i?'4i?retries=10'?$ddl_file_path/$f????????sed?-i?'5i?retry.backoff=3'?$ddl_file_path/$f????fidone??##?Func?2?-?如果文件夾不存在,則創建;反之提示文件已存在?if?[?!?-d?"beijing"?];then????mkdir?beijingelse????echo?"File?already?exists"fi
文件類型的參數判別大全:
shell腳本判斷文件類型?shell判斷文件,目錄是否存在或者具有權限?#!/bin/sh??myPath="/var/log/httpd/"?myFile="/var/log/httpd/access.log"??#?這里的-x?參數判斷$myPath是否存在并且是否具有可執行權限?if?[?!?-x?"$myPath"];?then?mkdir?"$myPath"??fi???#?這里的-d?參數判斷$myPath是否存在??if?[?!?-d?"$myPath"];?then??mkdir?"$myPath"??fi???#?這里的-f參數判斷$myFile是否存在??if?[?!?-f?"$myFile"?];?then??touch?"$myFile"??fi???#?其他參數還有-n,-n是判斷一個變量是否是否有值??if?[?!?-n?"$myVar"?];?then??echo?"$myVar?is?empty"??exit?0??fi???#?兩個變量判斷是否相等??if?[?"$var1"?=?"$var2"?];?then??echo?'$var1?eq?$var2'??else??echo?'$var1?not?eq?$var2'??fi
文件的判別邏輯大全:
-a?file?exists.?-b?file?exists?and?is?a?block?special?file.?-c?file?exists?and?is?a?character?special?file.?-d?file?exists?and?is?a?directory.?-e?file?exists?(just?the?same?as?-a).?-f?file?exists?and?is?a?regular?file.?-g?file?exists?and?has?its?setgid(2)?bit?set.?-G?file?exists?and?has?the?same?group?ID?as?this?process.?-k?file?exists?and?has?its?sticky?bit?set.?-L?file?exists?and?is?a?symbolic?link.?-n?string?length?is?not?zero.?-o?Named?option?is?set?on.?-O?file?exists?and?is?owned?by?the?user?ID?of?this?process.?-p?file?exists?and?is?a?first?in,?first?out?(FIFO)?special?file?or?named?pipe.?-r?file?exists?and?is?readable?by?the?current?process.?-s?file?exists?and?has?a?size?greater?than?zero.?-S?file?exists?and?is?a?socket.?-t?file?descriptor?number?fildes?is?open?and?associated?with?a?terminal?device.?-u?file?exists?and?has?its?setuid(2)?bit?set.?-w?file?exists?and?is?writable?by?the?current?process.?-x?file?exists?and?is?executable?by?the?current?process.?-z?string?length?is?zero.??注意-s?和?-f?參數的區別
cat?-n?file1?標示文件的行數?cat?example.txt?|?awk?'NR%2==1'?刪除example.txt文件中的所有偶數行?echo?a?b?c?|?awk?'{print?$1}'?查看一行第一欄?echo?a?b?c?|?awk?'{print?$1,$3}'?查看一行的第一和第三欄?paste?file1?file2?合并兩個文件或兩欄的內容?paste?-d?'+'?file1?file2?合并兩個文件或兩欄的內容,中間用"+"區分?sort?file1?file2?排序兩個文件的內容?sort?file1?file2?|?uniq?取出兩個文件的并集(重復的行只保留一份)?sort?file1?file2?|?uniq?-u?刪除交集,留下其他的行?sort?file1?file2?|?uniq?-d?取出兩個文件的交集(只留下同時存在于兩個文件中的文件)?comm?-1?file1?file2?比較兩個文件的內容只刪除?'file1'?所包含的內容?comm?-2?file1?file2?比較兩個文件的內容只刪除?'file2'?所包含的內容?comm?-3?file1?file2?比較兩個文件的內容只刪除兩個文件共有的部分
綜合案例:
①:本機tcp各種狀態數統計
netstat?-n?|?awk?'/^tcp/?{++S[$NF]}?END?{for(a?in?S)?print?a,?S[a]}'CLOSE_WAIT?2ESTABLISHED?276SYN_SENT?2TIME_WAIT?63270
awk ‘BEGIN { sum = 0; for (i = 0; i 50) exit(10); else print “Sum =”, sum } }’
②:使用cut提取文本字符串:
按“:”分割,提取/etc/passwd中第1,3,4,5列數據,按原分隔符”:”進行拼接。
[hdfs@cdh01?test]$?head?-n?5?/etc/passwd?|?cut?-d?:?-f?1,3-5root:0:0:rootbin:1:1:bindaemon:2:2:daemonadm:3:4:admxixi:4:7:xixi
③:使用uniq進行濾重
cut的具體用法,如下:
A).去除重復行
sort?target_file?|?uniq
B).查找非重復行
sort?target_file?|?uniq?-u
C).查找重復行
sort?target_file?|?uniq?-d
D).統計每一個文件出現的次數
sort?target_file?|?uniq?-c
④:排序
sort linux下的排序工具
參數說明:
-
-r 降序排列 -
-u 去重 -
-n 以數字大小排序(默認是首字母排序) -
-t 指定分隔符 這里我們指定’|’為分隔符 -
-k 指定分隔后的第幾位進行排序 這里我們指定第2位
任務:report.txt文件里有以下內容:記錄了一些方法的執行時間,要求按執行時間降序排列.
命令:sort -run -k 2 -t ‘|’ report.txt
?1?void?com.dustpan.zeus.core.service.MergeService.startService(int)|2?2?void?com.dustpan.zeus.core.service.InitShopDateService.startService(int)|1?3?boolean?com.dustpan.zeus.core.service.MergeService.executeGePrintSyncTask()|475?4?boolean?com.dustpan.zeus.core.service.InitShopDateService.executeInitShopTask()|96?5?boolean?com.dustpan.zeus.core.service.MergeService.executeGePrintSyncTask()|1013?6?boolean?com.dustpan.zeus.core.service.InitShopDateService.executeInitShopTask()|184?7?boolean?com.dustpan.zeus.core.service.MergeService.executeGePrintSyncTask()|729?8?boolean?com.dustpan.zeus.core.service.InitShopDateService.executeInitShopTask()|14?9?boolean?com.dustpan.zeus.core.service.MergeService.executeGePrintSyncTask()|39410?boolean?com.dustpan.zeus.core.service.InitShopDateService.executeInitShopTask()|9011?boolean?com.dustpan.zeus.core.service.MergeService.executeGePrintSyncTask()|56912?boolean?com.dustpan.zeus.core.service.InitShopDateService.executeInitShopTask()|79613?boolean?com.dustpan.zeus.core.service.MergeService.executeGePrintSyncTask()|164814?boolean?com.dustpan.zeus.core.service.InitShopDateService.executeInitShopTask()|8215?boolean?com.dustpan.zeus.core.service.MergeService.executeGePrintSyncTask()|101816?boolean?com.dustpan.zeus.core.service.InitShopDateService.executeInitShopTask()|1417?boolean?com.dustpan.zeus.core.service.MergeService.executeGePrintSyncTask()|93718?boolean?com.dustpan.zeus.core.service.InitShopDateService.executeInitShopTask()|1719?boolean?com.dustpan.zeus.core.service.MergeService.executeGePrintSyncTask()|60120?boolean?com.dustpan.zeus.core.service.InitShopDateService.executeInitShopTask()|5221?boolean?com.dustpan.zeus.core.service.MergeService.executeGePrintSyncTask()|508122?boolean?com.dustpan.zeus.core.service.InitShopDateService.executeInitShopTask()|38823?boolean?com.dustpan.zeus.core.service.MergeService.executeGePrintSyncTask()|19824?boolean?com.dustpan.zeus.core.service.InitShopDateService.executeInitShopTask()|1125?boolean?com.dustpan.zeus.core.service.MergeService.executeGePrintSyncTask()|20326?boolean?com.dustpan.zeus.core.service.InitShopDateService.executeInitShopTask()|1127?boolean?com.dustpan.zeus.core.service.MergeService.executeGePrintSyncTask()|24128?boolean?com.dustpan.zeus.core.service.InitShopDateService.executeInitShopTask()|1329?boolean?com.dustpan.zeus.core.service.MergeService.executeGePrintSyncTask()|17630?boolean?com.dustpan.zeus.core.service.InitShopDateService.executeInitShopTask()|1231?boolean?com.dustpan.zeus.core.service.MergeService.executeGePrintSyncTask()|20632?boolean?com.dustpan.zeus.core.service.InitShopDateService.executeInitShopTask()|3333?boolean?com.dustpan.zeus.core.service.MergeService.executeGePrintSyncTask()|24234?boolean?com.dustpan.zeus.core.service.InitShopDateService.executeInitShopTask()|19
案例一、找出本文件夾中包含my_test_calendar關鍵字的所有文件(含全路徑+文件名),并濾重。
查詢hive_sh目錄下,包含搜索關鍵字’my_test_calendar’的文本出現的文件及關鍵字所在的行,將文件(文件路徑+文件名) 字段提取出來,并做濾重處理。
[hdfs@nn1?hive_sh]$?find?.?-type?f?|?xargs?grep?-rn?'my_test_calendar'?|?cut?-d?:?-f?1?|?uniq
案例二、查找所有出現指定文本的文件并濾重(進階版)
腳本說明:
遍歷 hive_tables.txt 文件中的所有表(如 hive_table_1),查找其在指定目錄/data/program/hive_sh/下的文件中,是否使用到。如果查到重復出現該文本的文件,做濾重處理。
cat query_table_usage_in_hive.sh
#!/bin/bash?while?read?tbldo?echo?------------------Handle?table:?$tbl--------------find?/data/program/hive_sh/?-type?f?|?xargs?grep?-rn?$tbl?|?cut?-d?:?-f?1?|?uniq?>?./output/${tbl}_result.txt?done?<p data-tool="mdnice編輯器" style="margin-bottom: 0px;padding-top: 8px;padding-bottom: 8px;outline: 0px;letter-spacing: 0.544px;white-space: normal;text-size-adjust: auto;color: rgb(0, 0, 0);font-family: Optima-Regular, Optima, PingFangSC-light, PingFangTC-light, " pingfang sc cambria cochin georgia times new roman serif left>案例三、查找內存耗用top3的app</p><pre class="brush:php;toolbar:false;">ps?auxw?|?head?-1;ps?auxw|sort?-rn?-k4|head?-5