FREE THOUGHT · FREE SOFTWARE · FREE WORLD

MariaDB with LZ4 Compression Howto for CentOS

I really really wanted to run the latest MariaDB with LZ4 Page Compression.. it is a game changer for many types of large databases I deal with. So I compiled it manually in a way that is repeatable and follows best-practices and it's now powering this site. Now I can use InnoDB Page Compression with lzo, lzma, bzip2, snappy, or my favorite algorithm: LZ4.

Thought this would be a good chance to post a howto, and show that there is a lot you can do by compiling software yourself and breaking the package management one-click install shackles.

Crazy POWERFUL Bash Prompt

bash power prompt PS1This amazing bash linux prompt does more than meets the eye. If you want to know how to become really good with technology, linux is the secret sauce behind the AskApache articles. Open Source is elixir of the web. Thanks to everyone who helped me for the past 20 years. I use linux/bsd because homey don't play, so this is geared to be as productive a prompt as I can make it.

Don't have much time.. or just don't care? Not a problem, here are the 3 lines to copy and paste - you can just paste them right in your shell to test it, or add to a startup script.

export AA_P="export PVE=\"\\033[m\\033[38;5;2m\"\$(( \`sed -n \"s/MemFree:[\\t ]\\+\\([0-9]\\+\\) kB/\\1/p\" /proc/meminfo\` / 1024 ))\"\\033[38;5;22m/\"\$((\`sed -n \"s/MemTotal:[\\t ]\\+\\([0-9]\\+\\) kB/\\1/p\" /proc/meminfo\`/ 1024 ))MB\"\\t\\033[m\\033[38;5;55m\$(< /proc/loadavg)\\033[m\";echo -en \"\""
export PROMPT_COMMAND="history -a;((\$SECONDS % 10==0 ))&&eval \"\$AA_P\";echo -en \"\$PVE\";"
export PS1="\\[\\e[m\\n\\e[1;30m\\][\$\$:\$PPID \\j:\\!\\[\\e[1;30m\\]]\\[\\e[0;36m\\] \\T \\d \\[\\e[1;30m\\][\\[\\e[1;34m\\]\\u@\\H\\[\\e[1;30m\\]:\\[\\e[0;37m\\]\${SSH_TTY} \\[\\e[0;32m\\]+\${SHLVL}\\[\\e[1;30m\\]] \\[\\e[1;37m\\]\\w\\[\\e[0;37m\\] \\n(\$SHLVL:\\!)\\\$ " && eval $AA_P

TMUX

penguin-tmuxarchlinux-on-macStop wasting your lives with Mac Terminals.. or Macs. Get a real machine and then get a real shell multiplexer! For many years we all loved GNU Screen, but tmux is by far a better option today. The only time I am in the shell and not using a multiplexer, is when I'm not on one of my machines. My Arch Linux machines all run URxvt and my .bash_profiles all start tmux automataically, whether in X or single-user mode, tmux is where it's at.

Linux Debugging using a Bootloader with Kernel Parameters

grub awesomeAs an example, here is the boot line that I am using at the moment on an older Dell Desktop, just to illustrate module parameters and environment vars.

title  Arch Linux X-256
linux   /vmlinuz-linux root=UUID rw rootfstype=ext4 elevator=noop selinux=0 plymouth.enable=0 raid=noautodetect ipv6.disable=1 video=DP1:d rcutree.rcu_idle_gp_delay=1 i915.panel_ignore_lid=-1 i915.modeset=1 TERM=rxvt-256color

RXVT Customization with ~/.Xresources

rxvt-unicode screenshot running tmuxAfter many years of using any and all terminal emulators out there, from xterm to the Gnome terminal, to KDE Konsole to xfce4-terminal, lxterminal, vte, yakuake, rote, roxterm, sakura, terminator, and putty I finally I settled in for the long-haul with rxvt (rxvt-unicode).
My BOX: Slim -> Ratpoison -> URxvt -> Tmux -> Bash

Building strace-plus

strace-plusstrace+ is an improved version of strace that collects stack traces associated with each system call. Since system calls require an expensive user-kernel context switch, they are often sources of performance bottlenecks. strace+ allows programmers to do more detailed system call profiling and determine, say, which call sites led to costly syscalls and thus have potential for optimization.

Fix Linux Users Home Permissions with a Cron Job

As a security nut myself, and also a Linux admin, one of my biggest pet peeves is when I've taken the time and care to segment all the users on a server into separate home directories, and then some developer comes along, logs in as root, and changes the ownership of files. Other things can cause this, like Apache, PHP, Mutt, etc.. So I've always used a cron job that executes daily (and on demand) which automatically fixes all the permissions back to what they should be.

Install multiple OS Without Cds

Orig published 2006. I had a CD-RW drive but being a computer security researcher I had no money for blank cd-recordables. What follows is how I managed to install various operating systems on my computer (1 hard drive) without having to burn to a CD the ISO and then boot from that.

And also:

==Phrack Inc.==
 
              Volume 0x0b, Issue 0x3f, Phile #0x0a of 0x14
 
|=-----------------=[ Hacking Grub for fun and profit ]=-----------------=|

Optimized Vimrc with 256 Colorscheme

askapachecode 256color colorscheme and vimrc screenshotThis is not your everyday vimrc. This is my personalized vimrc.. works especially great on remote servers, in tmux, screen, etc.

This doesn't require a git checkout, a vim update, a bash update, an OS update, a plugin. No. 1 vimrc, 1 colorscheme, just use curl to download and it's on. I use this on Arch Linux, NetBSD, FreeBSD, OpenBSD, Debian, Ubuntu, Cent OS, Red Hat, and that's mostly it so far.
If you aren't already using vim as your primary editor in a powerful way, you are gonna owe me for this.

ASCII Chart

Wanted to stick this here for a reference, mostly for me. I use ASCII alot in bash, preg_matches, preg_replace, etc..

Custom bash_profile for Advanced Shell Users

Looking for some advanced uses for the shell? Here is some of my best. The shell is where 70% of my work takes place, and I have at least one terminal open almost 100% of the time, for viewing tailing color-coded logs, and of course for the SSH Tunnels that I use to route various networking through, like my email. So I decided that to standardize and create a bash_profile containing the most time-saving and helpful functions that I could use on all the various hosting environments would really be some sweet sugar, so here is my constant Work-in-progress.

It works for all shells I encounter, including BackTrack, Debian, Knoppix, Arch Linux, etc. Also works for many hosting environments I use including DreamHost, HostGator, WiredTree, and pretty much any linux VPS.

I also rely on this heavily from within shell scripts I write to access all the functions and stuff in this .bash_profile, and to do that I just do like:

#!/bin/bash

source ~/.bash_profile &>/dev/nulll

pm "PM is a function to output nice messages with color"
yn "Are you enjoying the shell" && pm "Thats great!" || pm "Perhaps you're better suited for DOS"
yn "Show Calendar" && aa_calendar
yn "Show Fortune" && aa_fortune

Elite Log File Scrolling with Color Syntax

Scrolls the latest log entries for multiple log files to the current screen or to any other monitor or TTY in color using syntax highlighting, making debugging easier and saving a lot of time for multi-monitor workstations.

Awk Tutorial and Introduction

While researching a unix/linux tool awk I came upon one of the most thorough and helpful tutorials I've ever seen devoted to a particular topic. It's old-school just the way I like it. I contacted the author, Bruce Barnett because I just HAD to have this article for my readers, who are predominantly running solaris/unix/bsd/linux and he kindly gave permission.

Wget Trick to Download from Restrictive Sites

Before
wget 403 Forbidden After trick
wget bypassing restrictions
I am often logged in to my servers via SSH, and I need to download a file like a WordPress plugin. I've noticed many sites now employ a means of blocking robots like wget from accessing their files. Most of the time they use .htaccess to do this. So a permanent workaround has wget mimick a normal browser.


Linux

Linux Penguin Linux is a Unix-like computer operating system. Linux is one of the most prominent examples of free software and open source development; its underlying source code can be freely modified, used, and redistributed by anyone.

The Linux kernel was first released to the public on 17 September 1991, for the Intel x86 PC architecture. The kernel was augmented with system utilities and libraries from the GNU project to create a usable operating system, which later led to the alternate term GNU/Linux. Linux is now packaged for different uses in Linux distributions, which contain the sometimes modified kernel along with a variety of other software packages tailored to different requirements.

Predominantly known for its use in servers, Linux has gained the support of corporations such as IBM, Sun Microsystems, Dell, Hewlett-Packard and Novell, and is used as an operating system for a wide variety of computer hardware, including desktop computers, supercomputers, and embedded devices such as mobile phones and routers.

Unix

Unix (officially trademarked as UNIX®) is a computer operating system originally developed in 1969 by a group of AT&T employees at Bell Labs including Ken Thompson, Dennis Ritchie and Douglas McIlroy. Today's Unix systems are split into various branches, developed over time by AT&T as well as various commercial vendors and non-profit organizations.

As of 2007, the owner of the trademark UNIX® is The Open Group, an industry standards consortium. Only systems fully compliant with and certified to the Single UNIX Specification qualify as "UNIX®" (others are called "Unix system-like" or "Unix-like").

During the late 1970s and early 1980s, Unix's influence in academic circles led to large-scale adoption of Unix (particularly of the BSD variant, originating from the University of California, Berkeley) by commercial startups, the most notable of which is Sun Microsystems. Today, in addition to certified Unix systems, Unix-like operating systems such as Linux and BSD derivatives are commonly encountered.

Sometimes, "traditional Unix" may be used to describe a Unix or an operating system that has the characteristics of either Version 7 Unix or UNIX System V.

BSD

Berkeley Software Distribution Devil DaemonBerkeley Software Distribution (BSD, sometimes called Berkeley Unix) is the Unix derivative distributed by the University of California, Berkeley, starting in the 1970s. The term "BSD" is often non-specifically used to refer to any of the BSD descendants of today (e.g. FreeBSD, NetBSD, OpenBSD).

BSD is one of several branches of Unix operating systems. Another one is evolved from UNIX System V developed by AT&T's Unix System Development Labs. A third consists of the GNU/Linux operating systems which draw from both Unix System V and BSD, as well as Plan9, and non-UNIX operating systems.

BSD was widely identified with the versions of Unix available for workstation-class systems. This can be attributed to the ease with which it could be licensed and the familiarity it found among the founders of many technology companies during the 1980s. This familiarity often came from using similar systems—notably DEC's Ultrix and Sun's SunOS—during their education. While BSD itself was largely superseded by the System V Release 4 and OSF/1 systems in the 1990s (both of which incorporated BSD code), in recent years modified open source versions of the codebase (mostly derived from 4.4BSD-Lite) have seen increasing use and development.

OpenBSD

OpenBSD BlowFish OpenBSD is a Unix-like computer operating system descended from Berkeley Software Distribution (BSD), a Unix derivative developed at the University of California, Berkeley. It was forked from NetBSD by project leader Theo de Raadt in late 1995. The project is widely known for the developers' insistence on open source code and quality documentation; uncompromising position on software licensing; and focus on security and code correctness. The project is coordinated from de Raadt's home in Calgary, Alberta, Canada. Its logo and mascot is Puffy, a pufferfish.

OpenBSD includes a number of security features absent or optional in other operating systems and has a tradition of developers auditing the source code for software bugs and security problems. The project maintains strict policies on licensing and prefers the open source BSD licence and its variants—in the past this has led to a comprehensive licence audit and moves to remove or replace code under licences found less acceptable.

As with most other BSD-based operating systems, the OpenBSD kernel and userland programs, such as the shell and common tools like cat and ps, are developed together in a single source repository. Third-party software is available as binary packages or may be built from source using the ports collection.

The OpenBSD project currently maintains ports for 17 different hardware platforms, including the DEC Alpha, Intel i386, Hewlett-Packard PA-RISC, AMD AMD64 and Motorola 68000 processors, Apple's PowerPC machines, Sun SPARC and SPARC64-based computers, the VAX and the Sharp Zaurus.

FreeBSD

Berkeley Software Distribution Devil Daemon FreeBSD is a Unix-like free operating system descended from AT&T UNIX via the Berkeley Software Distribution (BSD) branch through the 386BSD and 4.4BSD operating systems. It runs on Intel x86 family (IA-32) PC compatible systems (including the Microsoft Xbox[1]), and also DEC Alpha, Sun UltraSPARC, IA-64, AMD64, PowerPC and NEC PC-98 architectures. Support for the ARM and MIPS architectures are under development.

FreeBSD is developed as a complete operating system. The kernel, device drivers and all of the userland utilities, such as the shell, are held in the same source code revision tracking tree (CVS). This is in contrast to other free operating systems such as Linux where the kernel, userland utilities and applications are developed separately and packaged together by other groups as Linux distributions.

As an operating system, FreeBSD is generally regarded as reliable and robust, and of the operating systems that accurately report uptime remotely,[2] FreeBSD is the most common free operating system listed in Netcraft's list[3] of the 50 web servers with the longest uptime. A long uptime also indicates that no crashes have occurred and that no kernel updates have been deemed necessary, as installing a new kernel requires a reboot and resets the uptime counter of the system.

NetBSD

NetBSD Flag NetBSD is a freely redistributable, open source version of the Unix-derivative BSD computer operating system. It was the second open source BSD variant to be formally released, after 386BSD, and continues to be actively developed. Noted for its portability and quality of design and implementation, it is often used in embedded systems and as a starting point for the porting of other operating systems to new computer architectures.