diff --git a/docs/color.md b/docs/color.md index 5a87184..885bd1f 100644 --- a/docs/color.md +++ b/docs/color.md @@ -1,19 +1,21 @@ # Colors ## Bash Colors + [Script to display all terminal colors](https://askubuntu.com/a/1044802) + ```sh msgcat --color=test ``` - - ## Putty Colors + We need to use `putty-256color` instead of `xterm-256color` or else the `Home` key is not working. [Putty shows prompt with no color, but Linux SSH can](https://superuser.com/a/1502895) In Putty, change Settings -> Connection > Data > Terminal-type string to: `putty-256color`. ["Emulate" 256 colors in PuTTY terminal](https://superuser.com/a/436928) + 1. Configure Putty In Settings > Windows > Colours there is a check box for "Allow terminal to use xterm 256-colour mode". @@ -26,12 +28,11 @@ if your server has a terminfo entry for `putty-256color`, typically in `/usr/sha The main thing here is to make the server use an available `Terminfo` entry that most closely matches the way `Putty` is configured. ### 24bit -* [Getting 24-bit color working in terminals](https://pisquare.osisoft.com/s/Blog-Detail/a8r1I000000GvXBQA0/console-things-getting-24bit-color-working-in-terminals) - - +* [Getting 24-bit color working in terminals](https://pisquare.osisoft.com/s/Blog-Detail/a8r1I000000GvXBQA0/console-things-getting-24bit-color-working-in-terminals) ## LSCOLORS Schemes + ```sh for theme in $(vivid themes); do echo "Theme: $theme"; @@ -42,6 +43,3 @@ done vivid generate one-light LSCOLORS=$(vivid generate one-light) ``` - - - diff --git a/docs/config_files.md b/docs/config_files.md index 6f925a1..7d00b44 100644 --- a/docs/config_files.md +++ b/docs/config_files.md @@ -3,7 +3,6 @@ * [Difference Between `.bashrc`, `.bash-profile`, and `.profile`](https://www.baeldung.com/linux/bashrc-vs-bash-profile-vs-profile) * [What are the functional differences between .profile .bash_profile and .bashrc](https://serverfault.com/questions/261802/what-are-the-functional-differences-between-profile-bash-profile-and-bashrc) - `.bash_profile` and `.bashrc` are specific to `bash`, whereas `.profile` is read by many shells in the absence of their own shell-specific config files. (`.profile` was used by the original Bourne shell.) `.bash_profile` or `.profile` is read by login shells, along with `.bashrc`; subshells read only `.bashrc`. (Between job control and modern windowing systems, `.bashrc` by itself doesn't get used much. If you use `screen` or `tmux`, screens/windows usually run subshells instead of login shells.) diff --git a/docs/doc.md b/docs/doc.md index 117ed37..7505afc 100644 --- a/docs/doc.md +++ b/docs/doc.md @@ -11,14 +11,14 @@ paste <(zcat OPUS-multiun-v1-eng-spa.spa.gz) <(zcat OPUS-multiun-v1-eng-spa.eng. 'BEGIN {OFS = FS} (split($1, a, " +")<5 && split($2, b, " +")< 5) {print $1, $2}' ``` - ## Sort + Sort according to a set of columns: + ```sh zcat FILE.gz | awk -F'\t' '!_[$4,$5]++' ``` - ## Where is the process running If there is a running process like `vim` that you would like to properly stop, you need to find in which `tmux` window it is running. @@ -27,6 +27,7 @@ To help figure this out, given the PID, you can ask `lsof` for its `CWD`. ```sh lsof -a -d cwd -p PID ``` + ``` COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME vim 7688 larkins cwd DIR 0,47 4096 154447160 /gpfs/projects/DT/mtp/models/HoC-Senate/corpora/spm/v2 @@ -48,6 +49,7 @@ awk \ uniq.TRAIN_2021-2016_${BIFILTER}.tsv \ > TRAIN_indev.tsv ``` + ```sh # Filter-out awk \ @@ -58,14 +60,12 @@ awk \ > TRAIN_notindev.tsv ``` - ### Filter-out Testset From Train ```sh grep --text --line-regexp --invert-match --fixed-strings --file=$testset_filename ``` - ## Seeded `shuf` ```sh @@ -77,14 +77,14 @@ function get_seeded_random { shuf -i1-100 --random-source=<(get_seeded_random 42) ``` - ## Broken Symlinks + ```sh find . -type l ! -exec test -e {} \; -print ``` - ## Refresh Bash's Cache + [How do I clear Bash's cache of paths to executables?](https://unix.stackexchange.com/a/5610) `bash` does cache the full path to a command. You can verify that the command you are trying to execute is hashed with the type command: @@ -104,18 +104,16 @@ Or just one entry: For additional information, consult help hash and man bash. - - ## BASH debugging + * [Bash debugging - Youtube](https://www.youtube.com/watch?v=9pbpevjuwmI) * `PS4` `export PS4='${BASH_SOURCE}:${LINENO}: ${FUNCNAME[0]}() - [${SHLVL},${BASH_SUBSHELL},$?] '` * `bash -x` * `bashdb` * `shellcheck` - - ## `lvim` + Find commands `:WhichKey`. This opens a popup and if you type the shortcut key you get submenus. @@ -123,9 +121,8 @@ This opens a popup and if you type the shortcut key you get submenus. `f` find a file. - - ## GNU parallel a la Spark + ```sh function desubtokenize {} export -f desubtokenize @@ -143,22 +140,22 @@ zcat --force train.gz \ > train.tok.gz ``` - ## Weather + [wttr.in - GitHub](https://github.com/chubin/wttr.in): The right way to check the weather Get the weather: + * `curl wttr.in/CityName` * `curl v2d.wttr.in/CityName` - ## Login Name + Find the full name of a user from its username. + ```sh lslogins | fzf ``` - - ## Python How to profile python's import statements. @@ -174,7 +171,6 @@ or: PYTHONPROFILEIMPORTTIME=1 myscript.python ``` - ## Disk Usage ### Tools @@ -189,12 +185,12 @@ PYTHONPROFILEIMPORTTIME=1 myscript.python * [pdu: Highly parallelized, blazing fast directory tree analyzer](https://github.com/KSXGitHub/parallel-disk-usage) * [tin-summer: Find build artifacts that are taking up disk space](https://github.com/vmchale/tin-summer) - -### View disk usage by filetype. +### View disk usage by filetype ```sh dust -t ``` + ``` 3.0K ┌── (others) │ █ │ 0% 2.0K ├── .BLEU │ █ │ 0% @@ -219,7 +215,6 @@ dust -t 3.9G ┌─┴ (total) │██████████████████████████████████████████████ │ 100% ``` - ## Activate This is an example of an `activate` script when you compile a tool by hand and you don't install it a common place. @@ -249,7 +244,6 @@ export CPLUS_INCLUDE_PATH=$SENTENCEPIECE_HOME/include${CPLUS_INCLUDE_PATH:+:$CPL export PKG_CONFIG_PATH=$SENTENCEPIECE_HOME/lib/pkgconfig${PKG_CONFIG_PATH:+:$PKG_CONFIG_PATH} ``` - ## Grep for Emojis [POSIX and Unicode character categories](https://github.com/Genivia/ugrep?tab=readme-ov-file#posix-and-unicode-character-categories) diff --git a/docs/fzf.md b/docs/fzf.md index 0cad05f..ff2a423 100644 --- a/docs/fzf.md +++ b/docs/fzf.md @@ -5,34 +5,32 @@ [tmux-fzf](https://github.com/sainnhe/tmux-fzf): Use fzf to manage your tmux work environment! * List of bindings - * prefix F To launch tmux-fzf, press `prefix` + `F` (Shift+F). - + * prefix F To launch tmux-fzf, press `prefix` + `F` (Shift+F). ## `fzf-git.sh` [fzf-git.sh](https://github.com/junegunn/fzf-git.sh): bash and zsh key bindings for Git objects, powered by fzf. * List of bindings - * CTRL-G CTRL-F for **F**iles - * CTRL-G CTRL-B for **B**ranches - * CTRL-G CTRL-T for **T**ags - * CTRL-G CTRL-R for **R**emotes - * CTRL-G CTRL-H for commit **H**ashes - * CTRL-G CTRL-S for **S**tashes - * CTRL-G CTRL-E for **E**ach ref (`git for-each-ref`) + * CTRL-G CTRL-F for **F**iles + * CTRL-G CTRL-B for **B**ranches + * CTRL-G CTRL-T for **T**ags + * CTRL-G CTRL-R for **R**emotes + * CTRL-G CTRL-H for commit **H**ashes + * CTRL-G CTRL-S for **S**tashes + * CTRL-G CTRL-E for **E**ach ref (`git for-each-ref`) > :warning: You may have issues with these bindings in the following cases: > > * CTRL-G CTRL-B will not work if > CTRL-B is used as the tmux prefix > * CTRL-G CTRL-S will not work if flow control is enabled, > CTRL-S will freeze the terminal instead - > * (`stty -ixon` will disable it) + > * (`stty -ixon` will disable it) > > To workaround the problems, you can use > CTRL-G *{key}* instead of > CTRL-G CTRL-*{KEY}*. * Inside fzf - * TAB or SHIFT-TAB to select multiple objects - * CTRL-/ to change preview window layout - * CTRL-O to open the object in the web browser (in GitHub URL scheme) - + * TAB or SHIFT-TAB to select multiple objects + * CTRL-/ to change preview window layout + * CTRL-O to open the object in the web browser (in GitHub URL scheme) diff --git a/docs/install.md b/docs/install.md index 465948b..adede54 100644 --- a/docs/install.md +++ b/docs/install.md @@ -1,5 +1,7 @@ # Install + # Unsloth.ai + On GPSC7 * Set proxies(http and https) and correct `tmp` directories in `.profile` diff --git a/docs/miller.md b/docs/miller.md index 6be6e8e..f195a0c 100644 --- a/docs/miller.md +++ b/docs/miller.md @@ -4,6 +4,7 @@ * [GitHub](https://github.com/johnkerl/miller): Miller is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON. ## Tabulate BLEU Scores + * Reading a csv dataframe * Write a nice table using bars * Print numbers with 2 decimal @@ -32,6 +33,7 @@ ```sh bzcat sentence_word_count_fr.tsv.bz2 | head -n 3 ``` + ``` id datetime wc sentence House/House/391/Debates/001/HAN001 2006-04-03 11:05:00.000000 49 The 38th Parliament ... @@ -61,6 +63,7 @@ bzcat sentence_word_count_fr.tsv.bz2 \ then \ reorder -f id,start,end,elapse ``` + ``` +------------------------------------+---------------------+---------------------+--------------------+ | id | start | end | elapse | @@ -85,11 +88,13 @@ jq --raw-output --compact-output \ then rename en_sum,#en_word,fr_sum,#fr_word,fr_count,#sentence \ | mlr --opprint --barred summary -a count,null_count,distinct_count,mean,min,median,max,stddev ``` + ``` {"date":"2022-10-24","fr":18,"en":15} {"date":"2022-10-24","fr":18,"en":18} {"date":"2022-10-24","fr":29,"en":28} ``` + ``` date=2022-10-24,#en_word=55951,#fr_word=59892,#sentence=2692 date=2022-10-25,#en_word=73587,#fr_word=79288,#sentence=3660 @@ -103,7 +108,6 @@ date=2022-10-26,#en_word=29726,#fr_word=32800,#sentence=1492 | #fr_word | 112 | 0 | 112 | 71302.02678571429 | 27863.87682146542 | 3150 | 76163 | 133257 | | #sentence | 112 | 0 | 112 | 3282.6160714285716 | 1318.6068832416186 | 128 | 3502 | 6373 | - ## Group per Object ```sh @@ -115,11 +119,13 @@ jq --raw-output --compact-output \ then rename en_sum,#en_word,fr_sum,#fr_word,fr_count,#sentence \ | mlr --opprint --barred summary -a count,null_count,distinct_count,mean,min,median,max,stddev ``` + ``` {"sitting":{"parliament":44,"session":1,"number":116},"fr":18,"en":15} {"sitting":{"parliament":44,"session":1,"number":116},"fr":18,"en":18} {"sitting":{"parliament":44,"session":1,"number":116},"fr":29,"en":28} ``` + ``` sitting.parliament=44,sitting.session=1,sitting.number=116,#en_word=55951,#fr_word=59892,#sentence=2692 sitting.parliament=44,sitting.session=1,sitting.number=117,#en_word=73587,#fr_word=79288,#sentence=3660 diff --git a/docs/nvim.md b/docs/nvim.md index 49d04e3..934c0a0 100644 --- a/docs/nvim.md +++ b/docs/nvim.md @@ -3,14 +3,12 @@ * [Popular Neovim Configurations](https://dotfyle.com/neovim/configurations/top) * [Christian Chiarulli's neovim config](https://github.com/ChristianChiarulli/nvim/) - ## Tips-And-Tricks * [Is it possible to disable lsp formatting temporarily?](https://www.reddit.com/r/neovim/comments/oo8jcu/is_it_possible_to_disable_lsp_formatting/) You can avoid all autocommands with `:noa w`, or ignore some (or all) for some time with `:set eventignore=BufWritePre`. * Replace `gq{motion}` by `gw{motion}` to format the lines that {motion} moves over. - ## LazyVim LazyVim is a collection of plugins and is built on top of [lazy.vim](https://github.com/folke/lazy.nvim). @@ -20,7 +18,6 @@ LazyVim is a Neovim setup powered by [💤 lazy.nvim](https://github.com/folke/l * [GitHub](https://github.com/LazyVim/LazyVim): Neovim config for the lazy * [GitHub - lazy.vim](https://github.com/folke/lazy.nvim): 💤 A modern plugin manager for Neovim - ## TreeSitter TreeSitter in `nvim` is used for better syntax highlighting. @@ -28,6 +25,7 @@ TreeSitter in `nvim` is used for better syntax highlighting. Tree-sitter is a parser generator tool and an incremental parsing library. It can build a concrete syntax tree for a source file and efficiently update the syntax tree as the source file is edited. Tree-sitter aims to be: + * General enough to parse any programming language * Fast enough to parse on every keystroke in a text editor * Robust enough to provide useful results even in the presence of syntax errors @@ -37,13 +35,11 @@ Tree-sitter aims to be: * [Documentation](https://tree-sitter.github.io/tree-sitter/) * [nvim-treesitter](https://github.com/nvim-treesitter/nvim-treesitter): Nvim Treesitter configurations and abstraction layer - ### nvim-treesitter-textobjects [github](https://github.com/nvim-treesitter/nvim-treesitter-textobjects): Syntax aware text-objects, select, move, swap, and peek support. - ### nvim-treesitter-context [github](https://github.com/nvim-treesitter/nvim-treesitter-context): @@ -53,9 +49,6 @@ A Vim plugin that shows the context of the currently visible buffer contents. It's supposed to work on a wide range of file types, but is probably most useful when looking at source code files. In most programming languages this context will show you which function you're looking at, and within that function which loops or conditions are surrounding the visible code. - - - ## Telescope ### telescope.nvim @@ -70,13 +63,11 @@ FZF sorter for telescope written in c. `fzf-native` is a `c` port of `fzf`. It only covers the algorithm and implements few functions to support calculating the score. - ## which-key.nvim [github](https://github.com/folke/which-key.nvim): 💥 Create key bindings that stick. WhichKey is a lua plugin for Neovim 0.5 that displays a popup with possible keybindings of the command you started typing. - ## gitsigns.nvim **Equivalent to VCSVimdiff** @@ -86,6 +77,7 @@ Git integration for buffers. Super fast git decorations implemented purely in Lua. Features + * Signs for added, removed, and changed lines * Asynchronous using luv * Navigation between hunks @@ -101,22 +93,18 @@ Features * Support for yadm * Support for detached working trees. - ## vim-illuminate [github](https://github.com/RRethy/vim-illuminate): illuminate.vim - (Neo)Vim plugin for automatically highlighting other uses of the word under the cursor using either LSP, Tree-sitter, or regex matching. - ## trouble.Nvim [github](https://github.com/folke/trouble.nvim): 🚦 A pretty diagnostics, references, telescope results, quickfix and location list to help you solve all the trouble your code is causing. - ## Language Sever Protocol (LSP) - ## mason.nvim [github](https://github.com/williamboman/mason.nvim): @@ -131,40 +119,34 @@ It runs everywhere Neovim runs (across Linux, macOS, Windows, etc.), with only a Packages are installed in Neovim's data directory (`:h standard-path`) by default. Executables are linked to a single `bin/` directory, which `mason.nvim` will add to Neovim's PATH during setup, allowing seamless access from Neovim builtins (shell, terminal, etc.) as well as other 3rd party plugins. -For a list of all available packages, see https://mason-registry.dev/registry/list. - +For a list of all available packages, see . ## noice.nvim [github](https://github.com/folke/noice.nvim): 💥 Highly experimental plugin that completely replaces the UI for messages, cmdline and the popupmenu. - ## mini.indentscopre [github](https://github.com/echasnovski/mini.indentscope): Neovim Lua plugin to visualize and operate on indent scope. Part of 'mini.nvim' library. - ## lualine.nvim [github](https://github.com/nvim-lualine/lualine.nvim): A blazing fast and easy to configure neovim statusline plugin written in pure lua. - ## LuaSnip [github](https://github.com/L3MON4D3/LuaSnip): Snippet Engine for Neovim written in Lua. - ## nvim-cmp [github](https://github.com/hrsh7th/nvim-cmp): A completion plugin for neovim coded in Lua. A completion engine plugin for neovim written in Lua. Completion sources are installed from external repositories and "sourced". - ## cmp-nvim-lsp [github](https://github.com/hrsh7th/cmp-nvim-lsp): @@ -179,49 +161,41 @@ As these candidates are sent on each request, adding these capabilities will bre `nvim-cmp` provides manually triggered completion that can replace omnifunc. See `:help cmp-faq` for more details. - ## cmp-buffer [github](https://github.com/hrsh7th/cmp-buffer): nvim-cmp source for buffer words. - ## cmp-path [github](https://github.com/hrsh7th/cmp-path): nvim-cmp source for filesystem paths. - ## cmp_luasnip [github](https://github.com/saadparwaiz1/cmp_luasnip): luasnip completion source for nvim-cmp - ## mini.pairs [github](https://github.com/echasnovski/mini.pairs): Neovim Lua plugin to automatically manage character pairs. Part of 'mini.nvim' library. - ## mini.surround [github](https://github.com/echasnovski/mini.surround): Neovim Lua plugin with fast and feature-rich surround actions. Part of 'mini.nvim' library. - ## mini.ai [github](https://github.com/echasnovski/mini.ai): Neovim Lua plugin to extend and create `a`/`i` textobjects. Part of 'mini.nvim' library. - ## mini.comment [github](https://github.com/echasnovski/mini.comment): Neovim Lua plugin for fast and familiar per-line commenting. Part of 'mini.nvim' library. - ## nvim-notify [github](https://github.com/rcarriga/nvim-notify): diff --git a/docs/perl.md b/docs/perl.md index d0b4912..32477c9 100644 --- a/docs/perl.md +++ b/docs/perl.md @@ -1,6 +1,7 @@ # Perl ## Remove Repeating Chinese Characters + ```sh perl -ple 'BEGIN{use utf8;use open qw(:std :utf8);} s/(\p{Han})\1{1,}/\1/gm' < translation.zho.word ``` diff --git a/docs/rsync.md b/docs/rsync.md index fcd2fce..5f73f9b 100644 --- a/docs/rsync.md +++ b/docs/rsync.md @@ -2,6 +2,7 @@ [rsync copy over only certain types of files using include option](https://stackoverflow.com/a/11111793) NOTE you MUST add `--include='*/'` to let rsync at least visit all directories. + ```sh rsync \ -Parzu \ @@ -19,6 +20,7 @@ rsync \ ``` [How to use Rsync to copy only specific subdirectories (same names in several directories)](https://stackoverflow.com/questions/15687755/how-to-use-rsync-to-copy-only-specific-subdirectories-same-names-in-several-dir) + ```sh rsync \ -F \ diff --git a/docs/slurm.md b/docs/slurm.md index 6490dac..55ac643 100644 --- a/docs/slurm.md +++ b/docs/slurm.md @@ -115,8 +115,6 @@ sacct --long --jobs= sacct -l -j ``` - - ## Cluster information See what the nodes really offer. @@ -162,7 +160,6 @@ ReservationName=maintenance3 StartTime=2024-02-03T14:30:00 EndTime=2024-02-05T14 Users=root Accounts=(null) Licenses=(null) State=INACTIVE BurstBuffer=(null) Watts=n/a ``` - ### Node's Specs Get node's specs. @@ -171,6 +168,7 @@ Get node's specs. sinfo --Node --responding --long sinfo -N -r -l ``` + | NODELIST | NODES | PARTITION | STATE | CPUS | S:C:T | MEMORY | TMP_DISK | WEIGHT | AVAIL_FE | REASON | |----------|-------|-------------|------------|--------|--------|--------|----------|--------|----------|--------| | cn101 | 1 | TrixieMain* | drained | 64 | 2:16:2 | 192777 | 0 | 1 | (null) | update | @@ -180,8 +178,8 @@ sinfo -N -r -l | cn119 | 1 | TrixieMain* | idle | 64 | 2:16:2 | 192777 | 0 | 1 | (null) | none | | cn125 | 1 | TrixieMain* | idle | 64 | 2:16:2 | 192777 | 0 | 1 | (null) | none | - ### Cluster Usage + Find the cluster usage per user. ```sh @@ -207,7 +205,6 @@ Usage reported in Percentage of Total trixie ebadia Ebadi dt 0.00% 0.00% ``` - ### Allocated Hostnames of a Multinode Job Get a list of hostnames allocated to a multi node job. @@ -223,7 +220,6 @@ cn103 cn104 ``` - ## GPSC5 ### SSH to a Worker Node @@ -254,7 +250,6 @@ srun \ /bin/bash -l ``` - ## GPSCC ### Script Example @@ -293,7 +288,6 @@ cd /home/tes001/DT/tes001/LJSpeech-1.1/PT2 srun everyvoice train text-to-spec --devices 2 --nodes 1 config/everyvoice-text-to-spec.yaml ``` - ### Sleeper Job Start a sleeper job diff --git a/docs/tmux.md b/docs/tmux.md index 6633443..571ba69 100644 --- a/docs/tmux.md +++ b/docs/tmux.md @@ -24,8 +24,8 @@ select-pane -T "title" To share a window between two sessions: `tmux link-window -s -t ` - ## Find a Panes Running a Command + find-window [-iCNrTZ] [-t target-pane] match-string (alias: findw) Search for a fnmatch(3) pattern or, with -r, regular expression match-string