Friday, October 25, 2013

Bing! Rewards Automation script: Unix Cron

I developed Bing! Rewards Automation script to automate the process of gaining Bing! rewards. Then I enhanced it to version 3.0 to automate the script execution and retries and to be notified on custom events. Now I want to share the recipe of full automation - that is when you don't run the script manually, but it is cronned in your favorite flavor of Unix/Linux.

For security reasons it's better to create a new user - a system user, so to say. The only purpose of that user will be to run the Bing! Rewards Automation script:
# useradd -mU bingrewards
useradd creates a new user named bingrewards (lower-case by convention), creates a home directory for it and also creates a group with the same name and includes the user into it.

Now do
# passwd bingrewards
To change the password.
Finally log into the user's account:
$ su bingrewards
Now you need to clone Bing! Rewards automation git repository:
$ git clone -- https://github.com/sealemar/BingRewards ~/bin
$ cd ~/bin
$ chmod 600 config.xml
$ vim config.xml
The first line clones git repo into /home/bingrewards/bin.
The third line changes permissions so that only user bingrewards and root can read and/or write config.xml.
With the last line you may want to edit your config.xml with your configuration using this documentation.

Finally, the last thing to do is:
$ mkdir -p ~/var/log/bingrewards
$ crontab -e
and paste this into your crontab:
PATH=/usr/bin:$PATH
LOCAL_CONFIG_DIR=/home/bingrewards/etc

0   1   *   *   *   sleep $(($RANDOM \% 120))m && python2 /home/bingrewards/bin/main.py 2>&1 | gzip > /home/bingrewards/var/log/bingrewards/`date "+\%Y-\%m-\%dT\%H:\%M:\%S"`.log.gz
Where the first line sets PATH environment variable, which consists of directories to traverse to find an executable. /usr/bin is where sleep, python2, gzip and date sit in my Linux distribution.
LOCAL_CONFIG_DIR is an environment variable which is used in notify/onScriptComplete.sh and notify/onScriptFailure.sh of the standard Bing! Rewards Automation script distribution (version 3.0). There sits a file of format:
$ cat ${LOCAL_CONFIG_DIR}/mailx.config.sh
RECEIVER=john.smith@gmail.com
SENDER=my.alerts@gmail.com
SENDER_PASSWORD="xxx"
SMTP_SERVER=smtp://smtp.gmail.com:587
Finally, the last line of crontab sets the cron job which runs at 1 am every day. The cron job consists of three steps:

  1. sleep up to 120 minutes. This sleep is introduced not to attract too much attention from Bing! and can be omitted. For more information see this blog post.
  2. run the script itself. 2>&1 says output STDERR to STDOUT that is if exception occurs it will be in STDOUT.
  3. gzip STDOUT and put it in /home/bingrewards/var/log/bingrewards/ creating a file like 2013-10-25T04:30:01.log.gz

    This last line essentially creates a log - just in case if there is something interesting which you may want to look up later. Gzip creates an archive to save the disc space, because it's not so likely that you want to have an uncompressed log. Also, consider zless, zcat, zgrep to work with logs. Vim can open gzipped text files as well.

    From crontab man:
    The entire command portion of the line, up to a newline or % character, will be executed by /bin/sh or by the shell specified in the SHELL variable of the cronfile. Percent-signs (%) in the command, unless escaped with backslash (\), will be changed into newline characters, and all data after the first % will be sent to the command as standard input.
    that is why % characters are escaped.

With all this randomness Bing! Rewards Automation script is almost completely undiscoverable. There is a random sleep between queries, a random sleep after completing one account and starting another and finally there is a random sleep every day before the script starts. All that together highly increases the odds against identifying the script as a bot by Bing!

Happy Binging :)

Friday, October 18, 2013

Bing! Rewards Automation version 3.0

Check out the new version of Bing! Rewards Automation script - version 3.0

The main reason behind a new major version was the fact that I'd been marketing Bing! Rewards Automation script as a script which can be easily croned in true OS or scheduled on Windows to fully automate the usage, but I'd never done that myself. And I'd never done that myself because I didn't have a piece of hardware which I could use as a server. But now, as I have Raspberry Pi, which is definitely one of the best $45 investments, I've been automating everything around. Check out how to monitor a server and how to send automatic alerts.

I thought for a while how would I want the script to work and came up with an idea that it should do at least two things: retry if it fails (who knows what Bing! folks are doing, sometimes the right choice is a simple retry) and notify me (preferably by email) on a condition.

Events

This new version 3.0 features both retries and notifications both of which can optionally be conditional. Say, i.e. you definitely want to retry if the script fails requesting Bing!, or you know for sure that some account earns you 16 points a day, now you may set up a rule to retry (a number of times) if it doesn't earn you that much on some reason. And you may want to get notified. How? Your choice. The notification mechanism works through shell hooks. You specify a command which should be run and an condition condition which should be met. And you may put some specifiers in the command which have special meaning. The final command can send you an email or SMS or trigger some other behavior - send a command to your home robot over Bluetooth :)

Retries and notifications are part of events. Here is a complete list of events:

EventDescription
onErrorDefines what the script should do when an error occurs during processing an account. That can be any kind of error which results in generating result/error_*.html file (i.e. authorization, inexpected server response, etc.)
onCompleteDefines how the script should behave when it completes processing an account.
onScriptCompleteA special event which occurs only once when the script finishes executing.
onScriptFailureA special event which occurs only once and if the script fails with exception some time after successfully loading the config.

Example of events:

        <onError>
            <retry interval="5" salt="3.5" count="3" />
            <notify cmd="/home/BingRewards/bin/BingRewards/notify/onError.sh %a %p %r %P %l %i %e" />
        </onError>
        <onComplete>
            <retry if="%p lt 16" interval="5" salt="3.5" count="3" />
            <notify if="%l gt 3000"
                    cmd='/home/BingRewards/bin/BingRewards/notify/onComplete_reached.sh %a %p %r %P %l %i "reached %l Life Credits"' />
            <notify if="%p ne 16"
                    cmd="/home/BingRewards/bin/BingRewards/notify/onComplete.sh %a %p %r %P %l %i" />
            <notify if="%P gt 475"
                    cmd="/home/BingRewards/bin/BingRewards/notify/onComplete.sh %a %p %r %P %l %i" />
        </onComplete>

Here the script will retry up to 3 times on error sleeping anywhere between 5 and 8.5 seconds between retries. If an error occurres after the third retry, the Bing! Rewards Automation notifies calling onError.sh substituting the right values instead of the specifiers (%Letter). Once the script completes processing an account it will retry if a number of points earned for that account is less than 16. Afterwards it will notify if Lifetime Credits is more than 3000. It will notify if it earned not exactly 16 points. It will notify if total rewards points available on the account is greater than 475 - at the time of writing this post it was the price of $5 Amazon.com gift card on Gold account.

Check out Configuration for Bing! Rewards Automation script for the full manual on how to configure Bing! Rewards Automation script.

If you want to set up email alerts - check my post about how to do that.

This is an email I get from the script:

onComplete: account =Live_n...@hotmail.com, points = 0, retries = 3, totalPoints = 201, lifeCredits = 1831, ifStatement = 'p ne 16'
onComplete: account = Live_a...@hotmail.com, points = 0, retries = 3, totalPoints = 210, lifeCredits = 550, ifStatement = 'p ne 16'
onComplete: account = Live_s....@hotmail.com, points = 0, retries = 3, totalPoints = 208, lifeCredits = 1838, ifStatement = 'p ne 16'
onComplete: account = Facebook_n...@gmail.com, points = 0, retries = 3, totalPoints = 355, lifeCredits = 7120, ifStatement = 'p ne 31'

Example of cron job

0   3   *   *   *   /home/BingRewards/bin/main.py > /dev/null 2>&1
Run every morning at 3 am.

Etc.

One more thing was to move between queries sleep interval and between queries sleep salt from bingRewards.py to config.xml for easier configuration.

Project's folder structure was redesigned:

/main.py and config.xml
pkg/all the rest of the code
notify/notification scripts
test/unit tests
results/ -> result/script results and errors

Finally, a new command line option was added:
$ ./main.py --version
Bing! Rewards Automation script: 
Version: 3.0
See 'version.txt' for the list of changes
This code is published under LGPL v3 
There is NO WARRANTY, to the extent permitted by law.

Developed by: Sergey Markelov

Related posts

Configuration for Bing! Rewards Automation script

This is a configuration manual for Bing! Rewards Automation config.xml

Example

<configuration>
    <general 
            betweenQueriesInterval="2.0" 
            betweenQueriesSalt="3.0"
            betweenAccountsInterval="40.0" 
            betweenAccountsSalt="45.0" />

    <accounts>
        <account type="Facebook" disabled="false">
            <login>john.smith@gmail.com</login>
            <password>xxx</password>
        </account>
        <account type="Facebook" disabled="true">
            <login>blabla@mail.com</login>
            <password>xxx</password>
        </account>
        <account type="Live" disabled="false">
            <login>gogo@hotmail.com</login>
            <password>xxx</password>
        </account>
        <account type="Live" disabled="false">
            <login>hoho@hotmail.com</login>
            <password>xxx</password>
        </account>
    </accounts>

    <events>
        <onError>
            <retry interval="5" salt="3.5" count="3" />
            <notify cmd="/home/BingRewards/bin/notify/onError.sh %a %p %r %P %l %i %e" />
        </onError>
        <onComplete>
            <retry if="%p lt 16" interval="5" salt="3.5" count="3" />
            <notify if="%l gt 3000"
                    cmd='/home/BingRewards/bin/notify/onComplete_reached.sh %a %p %r %P %l %i "reached %l Life Credits"' />
            <notify if="%p ne 16"
                    cmd="/home/BingRewards/bin/notify/onComplete.sh %a %p %r %P %l %i" />
            <notify if="%P gt 475"
                    cmd="/home/BingRewards/bin/notify/onComplete.sh %a %p %r %P %l %i" />

            <account ref="Live_gogo@hotmail.com">
                <retry if="%p lt 16" interval="5" salt="3.5" count="3" />
                <notify if="%l gt 750"
                        cmd='/home/BingRewards/bin/notify/onComplete_reached.sh %a %p %r %P %l %i "Gold with %l Life Credits"' />
                <notify if="%p ne 16"
                        cmd="/home/BingRewards/bin/notify/onComplete.sh %a %p %r %P %l %i" />
                <notify if="%P gt 475"
                        cmd="/home/BingRewards/bin/notify/onComplete.sh %a %p %r %P %l %i" />
            </account>
            <account ref="Facebook_john.smith@gmail.com">
                <retry if="%p lt 31" interval="5" salt="3.5" count="3" />
                <notify if="%l gt 10000"
                        cmd='/home/BingRewards/bin/notify/onComplete_reached.sh %a %p %r %P %l %i "reached %l Life Credits"' />
                <notify if="%p ne 31"
                        cmd="/home/BingRewards/bin/notify/onComplete.sh %a %p %r %P %l %i" />
                <notify if="%P gt 475"
                        cmd="/home/BingRewards/bin/notify/onComplete.sh %a %p %r %P %l %i" />
            </account>
        </onComplete>
        <onScriptComplete>
            <notify cmd="/home/BingRewards/bin/notify/onScriptComplete.sh mail" />
        </onScriptComplete>
        <onScriptFailure>
            <notify cmd="/home/BingRewards/bin/notify/onScriptFailure.sh mail" />
        </onScriptFailure>
    </events>
</configuration>

Man

Section (optional) general
==========================

    Defines general configuration

    Supported attributes:

        betweenQueriesInterval  - (optional) - (double) - how many seconds the script should wait between queries
                                             - Defaults to 1 second

        betweenQueriesSalt      - (optional) - (double) - up to how many seconds to wait on top of _betweenQueriesInterval_ (rand(0, salt))
                                             - Defaults to 3 seconds

        betweenAccountsInterval - (optional) - (double) - how many seconds the script should wait after completing an account and before loggining into the next account
                                             - Defaults to 30 seconds

        betweenAccountsSalt     - (optional) - (double) - up to how many seconds to wait on top of _betweenAccountsInterval_ (rand(0, salt))
                                             - Defaults to 35.5 seconds


Section (optional) [configuration.events]
=========================================

    Defines how the script should react on events.

    Supported events: { onError, onComplete, onScriptComplete, onScriptFailure }

    Note: this section and all its subsections are applied per account
          for events { onError, onComplete }
          and only once for events { onScriptComplete, onScriptFailure }


Section (optional) [configuration.events.onError]
=================================================

    Defines what the script should do when an error occurs during processing an account. That
    can be any kind of error which results in generating result/error_*.html file
    (i.e. authorization, unexpected server response, etc.)

    Supported subnodes:

        retry   - (optional) - see EVENTS.RETRY
        notify  - (optional) - see EVENTS.NOTIFY
        account - (optional) - see EVENTS.ACCOUNT

    Note: multiple _onError_ sections will be processed as one


Section (optional) [configuration.events.onComplete]
====================================================

    Defines how the script should behave when it completes processing an account.

    Supported subnodes:

        retry   - (optional) - see EVENTS.RETRY
        notify  - (optional) - see EVENTS.NOTIFY
        account - (optional) - see EVENTS.ACCOUNT

    Note: multiple _onComplete_ sections will be processed as one


Section (optional) [configuration.events.onScriptComplete]
==========================================================

    A special event which occurs only once when the script finishes executing. A good example of
    utilizing this event is:

        1. In _onError_ and _onComplete_ append information to a log file.
        2. In _onScriptComplete_ send that big log file over email.

    Supported subnodes:

        notify - (required) - see [configuration.events.onScriptComplete.notify]

    Note: multiple _onScriptComplete_ sections will be processed as one


Section (optional) [configuration.events.onScriptFailure]
==========================================================

    A special event which occurs only once and if the script fails with exception some time after
    successfully loading the config.

    Supported subnodes:

        notify - (required) - see [configuration.events.onScriptFailure.notify]

    Note: multiple _onScriptFailure_ sections will be processed as one


Section (optional) [configuration.events.onScriptComplete.notify]
=================================================================

    Describes what action to take on event.
    More than one _notify_ can be specified.

    Supported attributes:

        cmd - (required) - shell command to run on event

                Note: this attribute is different from EVENTS.NOTIFY.cmd
                    _Format Specifiers_ are not supported here.


Section (optional) [configuration.events.onScriptFailure.notify]
=================================================================

    Describes what action to take on event.
    More than one _notify_ can be specified.

    Supported attributes:

        cmd - (required) - shell command to run on event

                Note: this attribute is different from EVENTS.NOTIFY.cmd
                      _Format Specifiers_ are not supported here.

                Note: 1 argument will be passed to _cmd_ which is exception string


Section (optional) EVENTS.ACCOUNT
=================================

    Overrides the script notification behavior for an account. If the behavior is not
    overridden, global notification behavior is applied.

    Supported attributes:

        ref - (required) - a reference to an account of a format:

                                accountType_accountLogin

                           ex. Facebook_john.smith@gmail.com

    Supported subnodes:

        retry  - (optional) - see EVENTS.RETRY
        notify - (optional) - see EVENTS.NOTIFY


Section EVENTS.RETRY
====================

    Tells the script it should retry processing an account.

    Supported attributes:

        if       - (optional) - See EVENTS.{RETRY|NOTIFY}.IF
        interval - (required) - (double) - how many seconds the script should wait between retries
        salt     - (optional) - (double) - up to how many seconds to wait on top of interval (rand(0, salt))
        count    - (required) - (unsigned) - how many times to retry


Section EVENTS.NOTIFY
=====================

    Defines the notification behavior of the script. The notification will be triggered
    only once per all retries on an account.

    More than one _notify_ can be specified.

    Supported attributes:

        if  - (optional) - see EVENTS.{RETRY|NOTIFY}.IF

        cmd - (required) - an external command to execute on event. _Format Specifiers_ can be
                passed as arguments. See _Format Specifiers_
                subsection.


Section EVENTS.{RETRY|NOTIFY}.IF
================================

    This section sets a condition which must be evaluated to TRUE in order for the script
    to trigger retry behavior (see EVENTS.RETRY).

    Format:
    ~~~~~~~

        "specifier comparison_operator value"

        _if_ statement consists of three parts:

            specifier           - identifies which value of the script evaluated thus far
                or _lhs_          should be compared. See _Format Specifiers_.

            comparison_operator - how the _specifier_ should be compared to the _value_.
                or _op_           See _Comparison Operators_ below.

            value               - the value which the _specifier_ should be compared to
                or _rhs_


    Comparison Operators:
    ~~~~~~~~~~~~~~~~~~~~~

        As per http://www.w3.org/TR/2008/REC-xml-20081126/#syntax

            (<) must not appear in their literal form, except when used as markup delimiters,
            or within a comment, a processing instruction, or a CDATA section.

        thus, in order to be consistent, integer comparison operators from "man test" are used
        (with one exception that "-" sign is omitted).

        From: http://unixhelp.ed.ac.uk/CGI/man-cgi?test

            INTEGER1 -eq INTEGER2
            INTEGER1 is equal to INTEGER2

            INTEGER1 -ge INTEGER2
                INTEGER1 is greater than or equal to INTEGER2

            INTEGER1 -gt INTEGER2
                INTEGER1 is greater than INTEGER2

            INTEGER1 -le INTEGER2
                INTEGER1 is less than or equal to INTEGER2

            INTEGER1 -lt INTEGER2
                INTEGER1 is less than INTEGER2

            INTEGER1 -ne INTEGER2
                INTEGER1 is not equal to INTEGER2


    Example:
    ~~~~~~~~

        "%r ge 3"    - 3 or more retries were made
        "%p lt 50"   - points (earned during this session) is greater than or equal to 50
        "%l gt 1000" - lifetime credits is greater than 1000


Format Specifiers
=================

    This section describes format specifiers.

    First column  - specifier

    Second column - where the specifier is applicable to (as one capital letter type)

                    Supported types:

                        C - EVENTS.NOTIFY.cmd
                        E - events.onError.notify.cmd (that type includes everything from C type)
                        I - EVENTS.{RETRY|NOTIFY}.IF

    Third column  - description

    %a - C  - account name and type (ex. Facebook - john.smith@gmail.com)
    %e - E  - error, which caused that notification
    %i - C  - prints corresponding _if statement_ or "(None)"
    %l - CI - lifetime credits
    %p - CI - points earned during this session
    %P - CI - total reward points available
    %r - CI - number of retries made during this session

Saturday, October 12, 2013

Monitorix - a better way to monitor a server

Monitorix definitely is an easy and convenient way to monitor a server. Look at this pictures of how I monitor my Raspberry Pi:





It features an easy way to monitor daily, weekly, monthly and yearly periods.

Graphs include:
  • System load
  • Kernel usage
  • Filesystem usage
  • Disk I/O
  • Network (global and per service/port. And I want to emphasize that - per service and per port - very cool)
  • Users
  • Sensors
  • and a bunch more which I turned off :)
Every graph in there is clickable and gives you a bigger picture of one of your interest.

Why Monitorix is so cool?

Monirorix is cool because:
  • It's free and opensource. I (personally) don't care much about the later, but some of us like that
  • Once installed, it can be accessed from any web browser. I monitor my server even from phone.
  • It's lightweight
    • It doesn't require a web server and can be configured either to use a web-server or to run in a stand-alone mode
  • It shows the information in graphs which are configurable and easy to read
  • It can be configured to fire alarms on certain conditions (i.e. I use an alarm to send me an email once the System Load is above 75% for the last 15 minutes. The second alarm I use is when the free space on / reaches 75%)
  • It has a built-in Basic Access Authentication which is a very handy to have if combined with the access from the Internet
  • It can automatically send email reports (daily, weekly, monthly, early) with all the graphs in there

How does Monitorix work?

Monitorix is a bunch of Perl scripts. It can be installed either from the official web site or as a package on many modern Linux/Unix distributions.
It reads data from various places and saves it in database files. It reads frequently enough, though. And writes database files frequently enough as well. So, that's not good to keep on running on a hard drive or SSD all the time. The later wears off and the former is too slow to operate fast.
Here is the good news:

Monitorix causes too much of Disk I/O - I wanna optimize it

Anything-sync-daemon is your best friend there. A good fellow Graysky developed it for us and published on his GitHub page. Read up the documentation on Arch wiki - it is pretty straight forward. Few words - it works by pushing a desired directory into tmpfs, which is essentially, your RAM. That really sky rockets your Monitorix. It works orders of magnitude faster. And don't worry, your data survives reboots. When the daemon starts it copies a directory into the RAM and when it stops, it transfers the data back to the drive.

Too much RAM is consumed, I wanna optimize that

That is also achievable. Out of the box Monitorix eats more than 40Mb. While it may sound goofy for modern servers, it doesn't sound the same when you want to install it on a BeagelBone or Raspberry Pi like I did. After tuning my installation consumes only 19Mb.

To cut some memory off Monitorix which is never used on purpose back to the system you need to open your monitorix.conf:
# vim /etc/monitorix.conf
  • Go to <graph_enable> section and turn everything you don't want to see to "n" state.
  • For each of the sections which stood in "y" check man monitorix.conf or the official documentation and see if the section has a list associated with it, i.e. fs is a good example - the smaller the list, the smaller the resulting db file will be. Warning from man:
    WARNING: Every time the number of groups in this option changes, Monitorix will resize the fs.rrd file accordingly, removing all historical data.

Alerts

Alerts are implemented as hooks to shell commands. The end-point can be any, starting from sh, Python and ending wherever your imagination can bring you. Alerts are really easy to use and a good example is shown right here in Automated Email Alerts.

Finally

Finally, I'm giving away my monitorix.conf which I tuned for Raspberry Pi. Pay extra attention to alerts subsection under system and under fs which you most likely will want to change to your own hooks.

Wednesday, October 2, 2013

SSH - a life behind Screen

Screen Screen is a full-screen window manager that multiplexes a physical terminal between several processes, typically interactive shell.

This post is not only SSH related, but it can be very nicely utilized while working with the local terminal. It is good when you're on Windows host with Putty and don't want to create enormous amount of windows. Also this is very useful, if you don't want to keep your ssh session open while a big chunk of job is being executed.

Here is a picture of my screen session:


Some points to pay attention to:

The very last line belongs to screen. It says I have 5 sessions (or tabs, or windows, pick your name).

users:1(5)1 user with 5 open sessions
uptime:2.02 days of uptime
mem:387megabytes free
cpu:18.17cpu load
Mon, 08/30date and time (truncated because of the size of the window)

All what you see on the last line is configurable and can be set to show different values. Say, for example, you might want to see your wifi status, or the CPU temperature.

All you need to do to get this is to install screen, and insert this into your ~/.screenrc


  1. # ~/.screenrc
  2. altscreen on                # Return screen to original state when running
  3.                             # ANSI programs like less and vi
  4. # turn off startup message
  5. startup_message off
  6. # detach on hangup
  7. autodetach on
  8. # emulate .logout message
  9. pow_detach_msg "Screen session of \$LOGNAME \$:cr:\$:nl:ended."
  10. # Add stuff to xterm (and cousins) title bars.  This is a moderate abuse of the
  11. # hardstatus feature--it just puts the hardstatus stuff into an xterm title
  12. # bar.
  13. setenv DISPLAY :0
  14. setenv TERM screen-256color
  15. backtick 1  10   0 /home/sealemar/etc/bin/screen/free.sh mem
  16. backtick 2  10   0 /home/sealemar/etc/bin/screen/free.sh swap
  17. backtick 3   7   7 /home/sealemar/etc/bin/screen/cpu_load.py
  18. backtick 4 300   0 /home/sealemar/etc/bin/screen/uptime.sh
  19. backtick 5  60   0 /home/sealemar/etc/bin/screen/date_time.sh
  20. backtick 6  15   0 /home/sealemar/etc/bin/screen/users_logged_in.sh
  21. #backtick 4  15  15 /home/sealemar/etc/bin/battery_life.pl --nocolor
  22. #backtick 4  30  30 /home/sealemar/etc/bin/screen/wifi.sh
  23. # Set the caption on the bottom line
  24. caption always
  25. #caption string "%{= kw}%-w%{= BW}%n %t%{-}%+w %-= wifi:%4` mem:%1` swap:%2` cpu:%3` - %LD %d %LM %Y - %c"
  26. caption string "%{= kw}%-w%{= BW}%n %t%{-}%+w %-= users:%6` uptime:%4`d mem:%1` swap:%2` cpu:%3` - %5`"
  27. # use xterm scrollback mechanism
  28. info xterm* ti@:te@
  29. # 256 color xterm
  30. attrcolor b ".I"    # allow bold colors - necessary for some reason
  31. # don't use visual bell
  32. vbell off
  33. # replace ctrl-A by ctrl-O
  34. escape ^Oo
  35. # set a big scrolling buffer
  36. defscrollback 20000
  37. hardcopydir ~/.cache/screen
  38. # set how screen will highlight the text
  39. sorendition =r
  40. # fix the residual text
  41. #altscreen on
  42. bind ' ' windowlist -b
  43. # resizing
  44. bind = resize =
  45. bind + resize +1
  46. bind - resize -1
  47. bind _ resize max
  48. bind ^A eval "split" "focus" "next" # ^O-^A to split the window into a new region
  49. bind ^X remove                      # ^O-^X to remove the current region
  50. bindkey -k k5 focus                 # F5 to cycle through split regions
  51. bindkey -k k6 prev          # F6 for previous window
  52. bindkey -k k7 next          # F7 for next window
  53. bindkey -k k8 copy          # F8 to enter scroll mode
  54. # create some screens
  55. screen -t booo1 1 bash
  56. screen -t booo2 2 bash
  57. screen -t booo3 3 bash
  58. screen -t booo4 4 bash
  59. screen -t Build 5 bash

Visible stuff is configured on lines 71-76 (sessions and their names which are created when screen start up) and lines 21-32 (counters, date time, etc.) I'm not going to elaborate on what every line does. The most interesting to tune, however, is backticks. It's obvious what it does from the documentation. Also, if you have any questions, don't hesitate to ask, I will do my best to answer everything.

Line #32 outputs a caption screen where user-defined backticks appear as a number between % and `, i.e "%6`"

The command below starts screen:
screen -dRR
which detaches screen session from wherever it was last opened from and attaches to it. If no screen session has been opened, a new one is created. Check man screen for a list of options, but that one above is the best I've found so far.

Here is a list of commands and shortcuts to use with the configuration above:

Command / ShortcutMeaning
screen -dRRDetaches existing screen session and attaches to it or starts a new session
F6Previous tab
F7Next tab
F8Select text
<Ctrl+O><Ctrl+]>Paste screen selection
<Ctrl+O><Ctrl+A>Split window
<Ctrl+O><Ctrl+X>Close split window
F5Cycle through split windows

One big plus of "ssh through screen" over "ssh without screen" is that you may start a job which takes hours to execute and then you can close your ssh client without worrying that the job will terminate and your session will end, because it won't. When you feel like you want to check how it goes, you just connect from the same place, or a different one, it doesn't matter, and attach to the existing screen session - that simple. If you are in the middle of something and you have a lot of stuff open, but you have to go right now and you don't want to close everything. It's all right, you don't have to, just close your ssh session. Next time, reattach to the screen with "screen -dRR" - done - everything is where you have left it.

Backticks

I publish all of my backticks with a right free to use, but without any warranties. References are appreciated.

date_time.sh


  1. /usr/bin/date +"%a, %D %H:%M

free.sh

  1. #!/bin/bash
  2. case "$1" in
  3.     "mem")  free -m | grep buffers/cache | awk '{print $4}' ;;
  4.     "swap") free -m | grep Swap | awk '{print $4}' ;;
  5. esac

cpu_load.py

  1. #!/usr/bin/env python2
  2. #
  3. # developed by Sergey Markelov (2013)
  4. import os, sys, time
  5. PIPE_WRITE_INTERVAL = 7
  6. PIPE_FILENAME = "/dev/shm/cpu_load.pipe"
  7. SINGLETON_FILE = "/dev/shm/cpu_load.py.singleton"
  8. def getStat():
  9.     with open("/proc/stat", "r") as fd:
  10.         times = fd.readline().split()[1:5]
  11.     return([int(t) for t in times])
  12. def cpuLoad(interval):
  13.     times1 = getStat()
  14.     time.sleep(interval)
  15.     times2 = getStat()
  16.     for i in range(len(times1)):
  17.         times2[i] -= times1[i]
  18.     usage = times2[0] + times2[1] + times2[2]
  19.     total = usage + times2[3]
  20.     return(100 * float(usage) / total)
  21. def writeData():
  22. # ensure this is the only instance
  23.     try:
  24.         open(SINGLETON_FILE, "r").close()
  25.     except IOError:
  26.         open(SINGLETON_FILE, "w").close()
  27.     else:
  28.         sys.exit(2)
  29.     try:
  30.         while(True):
  31.             load = cpuLoad(PIPE_WRITE_INTERVAL)
  32.             with open(PIPE_FILENAME, "w") as fd:
  33.                 fd.write(str(load))
  34.     except KeyboardInterrupt:
  35.         pass
  36.     finally:
  37.         os.remove(PIPE_FILENAME)
  38.         os.remove(SINGLETON_FILE)
  39. def readData():
  40.     try:
  41.         pipe = open(PIPE_FILENAME, "r")
  42.     except IOError:
  43.         print "N/A"
  44.     else:
  45.         print "%6.2f" % float(pipe.readline())
  46.         pipe.close()
  47. if __name__ == "__main__":
  48.     try:
  49.         if(len(sys.argv) < 2 or sys.argv[1] != "write"):
  50.             readData()
  51.         else:
  52.             writeData()
  53.     except Exception as e:
  54.         print("An unexpected exception has occured: %s" % str(e))
  55.         sys.exit(1)

uptime.sh


  1. #!/bin/sh
  2. # prints uptime as fraction of days
  3. # Example:
  4. # 3.5
  5. # means uptime is 3.5 days
  6. /usr/bin/awk '{printf "%0.1f\n", $1 / 3600 / 24}' /proc/uptime

users_logged_in.sh


  1. #!/bin/sh
  2. # prints
  3. # total_users(total_sessions)
  4. # Example:
  5. # 2(5)
  6. # means 2 users and 5 sessions in total
  7. PATH=/usr/bin
  8. users=`w -h | awk '{print $1}' | sort | uniq | wc -l`
  9. sessions=`w -h | awk '{print $1}' | wc -l`
  10. echo "${users}(${sessions})"