Saturday, February 1, 2014

BinaryClock Hardware

In the previous post I was talking about the software which runs on BinaryClock. Here I want to talk about the hardware. Here is how my version looks like. It doesn't mean if you want to assemble one you have to position all the parts in exact the same way.

BinaryClock default screen

Programming BinaryClock with FTDI cable

BinaryClock sliding decimal time
Schematics

Prototype video


You may find schematics and photos along with the source code of BinaryClock on my GitHub project page. The schematics and the photos are under doc/ folder. I hope those are pretty self-explanatory.

Instructions

The table below explains how push buttons control BinaryClock.

#ScreenDescriptionButtons behavior
1234
1 First Slides Welcome Message X (2) Show time in binary X X
2 Default Shows time in binary (3) Slide time in decimal (4) Set time (5) Show date in binary (6) Show events
3 Time in decimal Slides time from right to left (2) Show time in binary X (7) Slide date in decimal (6) Show events
4 Set time Set time in binary X Set next Decrease Increase
5 Date in binary Displays date in binary (7) Slide date in decimal (8) Set date (2) Show time (6) Show events
6 Show events Slides events from the next from today to the furthest from today (2) Show time (9) Show event year information Previous event Next event
7 Date in decimal Slides date from right to left (5) Show date in binary X (2) Show time (6) Show events
8 Set date Set date in binary X Set next Decrease Increase
9 Event year information Show event year information X Skip X X

Example: you are at screen #5 which displays date in binary. If you press the first button you will go to the screen #7 which slides date in binary. If you press the second button you will go to the screen #8 which sets the date. If you press the third button you will go to the screen #2 which displays time in binary. If you press the button number 4 you will go to the screen #6 which shows events. See the video above. More on events is on the next post.

How It Works

ATMEGA328p chip is what drives the BinaryClock. The chip runs the software. One 74HC595 (which is known as shift register) drives rows of the LED array, the other one drives columns. The output pins of the one which drives rows connect to the LED matrix through 260Ω resistors. 74HC595 which drives columns connects to LED array through ULN2803A which is also known as Darlington Transistor Array. Four push buttons are connected to ATMEGA328p which control the BinaryClock behavior.

Shift registers are there to reduce the number of pins used on ATMEGA328p. If two 74HC595 were not there, 16 pins on ATMEGA328p would be needed to achieve the same. Shift registers as the name suggests shift bits with each clock tick. The wiki will tell more about that. A good thing about shift registers is that they can be placed in series. So it doesn't matter how many shift registers you put in series, the same number of pins on the microcontroller can be used. Resistors are there to not over stress the LED matrix. If too much current goes through LED it burns brighter, but faster. Darlington array is there not to over stress the columns shift register. The current goes through the rows to the columns and further on. To light up an LED on the array the column needs to be grounded and the current should be applied to the row. You may notice that there are only 8 pins which drive rows and 8 pins which drive columns, so if you apply current to 8 rows and ground 8 columns, all the rows and columns will be lit. To display a pattern you will want to travel through the rows and display the columns or vise versa. More on the topic is here. BinaryClock software scans rows and uses walking pattern (as discussed on the last link) on columns. Thus at one point of time all the LEDs in a column may be lit and too much current will go to the columns pin. Darlington Array protects columns shift register from that over stress.

Finally in the top left corner you may notice legs. Those are legs to connect BinaryClock to FTDI which can be connected to a PC. In other words, that is the interface to program BinaryClock.

You may notice some other parts on the schematics, such as 16Mhz crystal oscillator, capacitors and a resistor on the first pin of ATMEGA328p. I'm not discussing those parts here. Please, refer to this page to better understand their purpose.

Optimizations & Improvements & Further thoughts

Maxim display driver may be used to reduce the number of parts significantly. If it was used instead, I wouldn't have two shift registers and Darlington Array on the board. And I would have only one resistor instead of eight (on rows) although it would be of a different value. Atmega chip has 32kb memory. It may seem like not a huge amount, but compilation of BinaryClock for it reports this:
AVR Memory Usage
----------------
Device: atmega328p

Program:   12306 bytes (37.6% Full)
(.text + .data + .bootloader)

Data:       1269 bytes (62.0% Full)
(.data + .bss + .noinit)
So, there are a lot of space left unused. You may want to extend the board with some sensors and then display the information from the sensors on the LED matrix.

Don't forget to edit lib/clock_event_personal.c with your personal events.

BinaryClock - a binary clock DIY

Idea

Here and there on the Internet you may see a geeky device called binary clock. The implementation can be different, but the idea behind is somewhat similar - it is an ordinary digital clock, but showing the time in binary format instead of decimal. After clicking the links you want one for yourself. Here I will tell you what you will need to create one with parts available at almost no cost on eBay.

There are a number of places where you can order a simple device which shows time. My idea was a clock which could show time, date and which could read decimal if I ever want to present that to somebody. Such a device wasn't out there, unfortunately, but looking retrospectively I'm positive it all was for the better because that led me to Arduino.

Arduino is an open-source electronics prototyping platform based on flexible, easy-to-use hardware and software. It's intended for artists, designers, hobbyists and anyone interested in creating interactive objects or environments.

Every Arduino device requires two things - the hardware and the software.

Functionality

The BinaryClock can:
  1. Show time in binary
  2. Slide time in decimal
  3. Show date in binary
  4. Slide date in decimal
  5. Set up time in binary
  6. Set up date in binary
  7. Slide user-defined events from the closest to the furthest
  8. Slide history information about an event such as when an event was first started and how many years the event is being celebrated
To better understand how it works, refer to the next post, where I discuss the hardware. Events are described here.

Software

Most of the time to write a program you don't have to have a device which it is intended to be used on. This code is no exception. Just by spending a little bit more time on design step a developer can get rid of a series of headaches later.

The source tree is organized in following way:

arduino/Arduino related code
doc/Documentation and schematic / PCB layout
emulator/PC emulator
include/A code which is common for arduino/emulator/lib/test
lib/BinaryClock library - the clock source
test/Tests
.ycm_extra_conf.pyI use YouCompleteMe which is a very handy plugin for ViM
LICENSEThe license
MakefileThe Makefile to build BinaryClock
project.vimA ViM project file - read on :)

The entire source code is published on my GitHub under Apache 2.0 license.

Tools I will need

This is a list of tools you will (or may) need to work with the code. This list will be described in more details further.
For the emulator and tests:
  • gcc / clang (at least one of these comes with every Unix based OS - Linux - gcc, MacOs - clang)
For the emulator:
  • ncurses (comes with every Unix based OS)
For the vimproject (optional - read on):
For the schematic:

Arduino

Under Arduino folder you will see only two files binary_clock.ino and Makefile.

binary_clock.ino - the code - is less than 200 lines long, half of which is comments. That's because all the functionality is hosted in lib/. This design makes easier testing and reusing the code (see emulator and test below). binary_clock.ino implements the code which is required by Arduino only, such as what pins the display sits on and what pins the buttons are on along with the glue code to bind the BinaryClock to the hardware.

Arduino website popularizes Arduino IDE. While it's ok for playing around, you may find yourself annoyed with the lack of functionality pretty fast. Luckily, there are ways to get around that. I personally like ViM. If you've never used ViM or Emacs - try one of them. Both have a very steep learning curve, but at the end of the day it worth every second. The next thing is Make. It can suit many needs. Just go read about it. To write a good Makefile from scratch is a challenge as the Make system has a very steep learning curve as well. And learning how to do it worth every second as well :). However, in many cases it's just easier to adopt an existing code, than to write your own, especially if it is well supported. And Makefile by Sudar / Martin Oldfield is a very good example of that. I took his Makefile as a basis.

Makefile under arduino folder of the project is definitions for Sudar's Makefile with some extra functionality. It has an error macro in the beginning which checks if Sudar's Makefile is installed. make will throw that error if it is missing.

To compile arduino code, go to arduino folder and run
$ cd BinaryClock/arduino
$ make
To upload the code to a device, run
$ make upload
If you use arduino as ISP programmer, run
$ make ispload
Note: if you don't have colorgcc installed, the Makefile may not work. If it doesn't, just comment two lines of code with "Override Arduino.mk defaults to point to my soft links" above them.

doc

Doc directory contains schematic which comes as a pdf, png and Fritzing project. And as far as the Fritzing project is concerned, there are two parts: FTDI Basic Male Header and Led Matrix LD1088-BS, which you may need to view the project. These parts are not needed for the pdf and the png, though.

emulator

As the name tells, the directory contains emulator related code. Like in the case of Arduino code, emulator is no more than the bindings and the glue code to work with the BinaryClock library (lib/).

Emulator uses ncurses library to emulate the clock in the terminal.


The emulator displays the time the same way it would be displayed on the real piece of hardware. The time follows the standard time representation on any electronic clock - from the left to the right one bar two pixels wide represents hours, minutes and seconds. And the one above reads 12:34:06. BinaryClock uses 24 hour format to eliminate ambiguity. Thus 12 is always 12pm. 12am would have 0 in the first column :)


One more example. On the emulator screen above it reads 23:59:55, which is 11:59pm.

To start the emulator, first you'll have to compile the code. From the project root do
$ make
This will compile everything - the library, Arduino code, the tests and the emulator. If you want to compile emulator only, do this:
$ cd BinaryClock/emulator
$ make
However it will still compile the library code, because it is the core of the BinaryClock. After the compiler has done its job, run
$ emulator/build/bin/terminal-binary-clock
From the project root. The emulator will show up. It will display a welcome message and then it will switch to the time screen. On the right you may notice numbers from 1 to 4 and the text further to the right after each of the numbers. The numbers are buttons. By pressing a corresponding button on the keyboard you execute a behavior associated with the button. On the real device, the buttons are simple push buttons. So, if you press 1, the emulator will show the time in text. BinaryClock has only 8x8 LEDs screen and the text is displayed by sliding it from right to left, the same way it slides on TV during the news hour.


The emulator reads the current time set on your PC when it starts, so you don't have to adjust it. On the real device the time most likely will need to be adjusted.

include

Include folder contains common functionality which is used from all other places. common.h and logger.h are included almost with every *.c unit. Makefile.include is included in the other Makefiles under lib, emulator, test.

lib

lib is a core of the BinaryClock. It is the code which BinaryClock runs. The modules are:

clockBinaryClock API
clock_alphabetCharacters which BinaryClock supports
clock_buttonFunctionality to work with the clock buttons
clock_eventEvents specific code
clock_event_personalThat's where you may set your own events which the clock will know of
clock_externA set of functions which need to be implemented by the user of the library. The pointers need to be assigned with the user-specific implementation
clock_mainThe only functionality to initialize and to update the clock state
clock_stateBinaryClock state object - the only object to pass around to the clock
clock_timeA mechanism to update BinaryClock time
date_timeInternal date-time code

The library is compiled into libclock.a which later can be referenced during the compilation of the binaries, such as emulator or test.

test

Contains a bunch of unit tests to test every external function from the lib. That proves that the BinaryClock would behave the way the developer wanted it to behave. test_ut is a light-weight test framework written especially for the project. It futures assert-like functionality which you may be familiar with from the modern test frameworks.

Run
$ cd BinaryClock
$make check
The example output will look like
--- main.c:48 (main) - a test suite started ---
Starting main.c:main --- [ut_clock]
--- ut_clock.c:252 (ut_clock) - a test suite started ---
Starting ut_clock.c:ut_clock --- [clock_slidePattern() returns correct result]
Starting ut_clock.c:ut_clock --- [clock_drawPattern() correct]
Starting ut_clock.c:ut_clock --- [clock_displayBinaryNumber() correct]
Starting ut_clock.c:ut_clock --- [clock_slideText() correct]
--- ut_clock.c:252 (ut_clock) - a test suite completed --- 4 test(s) ran. 4 succeded, 0 failed
Starting main.c:main --- [ut_clock_time]
--- ut_clock_time.c:76 (ut_clock_time) - a test suite started ---
Starting ut_clock_time.c:ut_clock_time --- [clock_updateUptimeMillis() returns correct delta]
Starting ut_clock_time.c:ut_clock_time --- [clock_updateUptimeMillis() handles integers overflows correctly]
--- ut_clock_time.c:76 (ut_clock_time) - a test suite completed --- 2 test(s) ran. 2 succeded, 0 failed
Starting main.c:main --- [ut_date_time]
...
 ...truncated...
...
Starting main.c:main --- [ut_clock_alphabet]
--- ut_clock_alphabet.c:67 (ut_clock_alphabet) - a test suite started ---
Starting ut_clock_alphabet.c:ut_clock_alphabet --- [clock_alphabet_getIndexByCharacter() correct]
Starting ut_clock_alphabet.c:ut_clock_alphabet --- [clock_alphabet_getIndexByCharacter() returns ERANGE and correct index]
--- ut_clock_alphabet.c:67 (ut_clock_alphabet) - a test suite completed --- 2 test(s) ran. 2 succeded, 0 failed
Starting main.c:main --- [ut_clock_event]
--- ut_clock_event.c:330 (ut_clock_event) - a test suite started ---
Starting ut_clock_event.c:ut_clock_event --- [clock_event_initDayOfMonth() is correct]
Starting ut_clock_event.c:ut_clock_event --- [clock_event_initDayOfWeek() is correct]
Starting ut_clock_event.c:ut_clock_event --- [clock_event_initDayOfYear() is correct]
Starting ut_clock_event.c:ut_clock_event --- [clock_event_getEventDetails() is correct]
Starting ut_clock_event.c:ut_clock_event --- [clock_event_initList() is correct]
Starting ut_clock_event.c:ut_clock_event --- [clock_event_updateList() is correct]
Starting ut_clock_event.c:ut_clock_event --- [clock_event_findClosestFromList() is correct]
Starting ut_clock_event.c:ut_clock_event --- [clock_event_yearInfoToStr() is correct]
--- ut_clock_event.c:330 (ut_clock_event) - a test suite completed --- 8 test(s) ran. 8 succeded, 0 failed
--- main.c:48 (main) - a test suite completed --- 6 test(s) ran. 6 succeded, 0 failed

project.vim

This you will only need if you are a ViM geek as I am.
$ alias gvimproject
alias gvimproject='gvim --cmd "source project.vim"'
I have the above alias in my environment and whenever I run gvimproject the shell starts an instance of GViM for me and preloads project.vim from the current folder.

This particular project.vim initializes path variable in ViM essentially telling it where the source code is located. So even if you work with the code under emulator folder and want to do gf (goto file) on a filename which refers to the code in lib, the Vim will know what you are trying to do. It also sets the appropriate ctags for you as you open up a *.c or an *.h file. It rebuilds the appropriate ctags file when you work with the code, so that when you press ctrl+] the tags are always fresh. And finally project.vim saves your current session when you leave Vim and automatically loads the last session when you start Vim. So, you don't have to remember which file was the last one that you worked with, what windows were opened, what line you were at, window size and position, etc.

Saturday, October 12, 2013

Monitorix - a better way to monitor a server

Monitorix definitely is an easy and convenient way to monitor a server. Look at this pictures of how I monitor my Raspberry Pi:





It features an easy way to monitor daily, weekly, monthly and yearly periods.

Graphs include:
  • System load
  • Kernel usage
  • Filesystem usage
  • Disk I/O
  • Network (global and per service/port. And I want to emphasize that - per service and per port - very cool)
  • Users
  • Sensors
  • and a bunch more which I turned off :)
Every graph in there is clickable and gives you a bigger picture of one of your interest.

Why Monitorix is so cool?

Monirorix is cool because:
  • It's free and opensource. I (personally) don't care much about the later, but some of us like that
  • Once installed, it can be accessed from any web browser. I monitor my server even from phone.
  • It's lightweight
    • It doesn't require a web server and can be configured either to use a web-server or to run in a stand-alone mode
  • It shows the information in graphs which are configurable and easy to read
  • It can be configured to fire alarms on certain conditions (i.e. I use an alarm to send me an email once the System Load is above 75% for the last 15 minutes. The second alarm I use is when the free space on / reaches 75%)
  • It has a built-in Basic Access Authentication which is a very handy to have if combined with the access from the Internet
  • It can automatically send email reports (daily, weekly, monthly, early) with all the graphs in there

How does Monitorix work?

Monitorix is a bunch of Perl scripts. It can be installed either from the official web site or as a package on many modern Linux/Unix distributions.
It reads data from various places and saves it in database files. It reads frequently enough, though. And writes database files frequently enough as well. So, that's not good to keep on running on a hard drive or SSD all the time. The later wears off and the former is too slow to operate fast.
Here is the good news:

Monitorix causes too much of Disk I/O - I wanna optimize it

Anything-sync-daemon is your best friend there. A good fellow Graysky developed it for us and published on his GitHub page. Read up the documentation on Arch wiki - it is pretty straight forward. Few words - it works by pushing a desired directory into tmpfs, which is essentially, your RAM. That really sky rockets your Monitorix. It works orders of magnitude faster. And don't worry, your data survives reboots. When the daemon starts it copies a directory into the RAM and when it stops, it transfers the data back to the drive.

Too much RAM is consumed, I wanna optimize that

That is also achievable. Out of the box Monitorix eats more than 40Mb. While it may sound goofy for modern servers, it doesn't sound the same when you want to install it on a BeagelBone or Raspberry Pi like I did. After tuning my installation consumes only 19Mb.

To cut some memory off Monitorix which is never used on purpose back to the system you need to open your monitorix.conf:
# vim /etc/monitorix.conf
  • Go to <graph_enable> section and turn everything you don't want to see to "n" state.
  • For each of the sections which stood in "y" check man monitorix.conf or the official documentation and see if the section has a list associated with it, i.e. fs is a good example - the smaller the list, the smaller the resulting db file will be. Warning from man:
    WARNING: Every time the number of groups in this option changes, Monitorix will resize the fs.rrd file accordingly, removing all historical data.

Alerts

Alerts are implemented as hooks to shell commands. The end-point can be any, starting from sh, Python and ending wherever your imagination can bring you. Alerts are really easy to use and a good example is shown right here in Automated Email Alerts.

Finally

Finally, I'm giving away my monitorix.conf which I tuned for Raspberry Pi. Pay extra attention to alerts subsection under system and under fs which you most likely will want to change to your own hooks.