sealemar talks 0x43 0x6F 0x64 0x65: coding

Showing posts with label coding. Show all posts

Friday, August 1, 2014

Браво! downloader

It turned out I like Russian Rock'n'Roll band 'Браво'. You may check it out here: Браво! official website.
The guys made all of the songs available for a free download. The quality is not so good, but hey, it's free.
I also found myself very lazy on clicking each and every link to download all of the tracks. If you feel the same - here is a Python script for your convenience which I've just developed: download.py.
It will download all of the tracks and put them under appropriate folder structure: album/track###.mp3
If it ever stops / fails to fetch a file, you may rerun the script - it is smart enough to understand what files have been downloaded so far, provided that they were not moved and the folder structure hasn't changed.
Enjoy!

Sunday, February 2, 2014

BinaryClock events

In two of my previous posts I was telling about a feature of the BinaryClock which I referred to as events. This is a very important feature which makes BinaryClock personal. It's better to describe it with example.

Example:

Say, today is December 23, 2013. The next event from today would be Christmas which is celebrated on December 25. If you are at the default screen which displays time in binary (refer to the table on this page) and you press button #4 (show event) then you'll see a sliding message "Chrismas - Dec 25 2013 Thursday", now if you press button #2 (show event year info) it will slide "2014 years - started in 0".

Another example:

Your friend Bob is having birthday tomorrow. Today is April 2, 2014. If you press button #4 (show event), you'll see a sliding message "Bob's birthday - Apr 3 2014" if you press button #2 while the event message is sliding, you'll see "25 years - started in 1989" (if Bob's turning 25).

Events can be set in three ways.

Month and day

Example is Christmas which is always celebrated on December 25. To add an event of this type, go to lib/clock_event_personal.c and add

clock_event_initDayOfMonth (25, DECEMBER, 0, "Christmas")

clock_event_initDayOfMonth() is a macro from lib/clock_event.h. Here is its prototype:

// @brief Initializer for an event which is set with month and day
// @param day day of month
// @param month month of the event
// @param year year when the event first occurred
// @param name name of the event
//
#define clock_event_initDayOfMonth(day, month, year, name)

Month and week

Thanksgiving in the US is celebrated on the fourth Thursday of November. To add an event of that type, go to lib/clock_event.personal.c and add

clock_event_initDayOfWeek (THURSDAY, 3, WEEK_FROM_START, NOVEMBER, 1574, "Thanksgiving")

clock_event_initDayOfWeek() is a macro from lib/clock_event.h

// @brief Initializer for an event which is set with
// day of week, week of month, and month
// @param dayOfWeek day of week. See macros in lib/date_time.h
// @param weekOfMonth week of the month. [0..3]
// @param fromBeginningOfMonth whether the _weekOfMonth_ should be counted from the beginning of the month
//          This can be any number or TRUE / FALSE which can be treated in the boolean context
//
// @param month month of the event
// @param year year when the event first occurred
// @param name name of the event
//
#define clock_event_initDayOfWeek(dayOfWeek, weekOfMonth, fromBeginningOfMonth, month, year, name)

weekOfMonth is a number from 0 to 3. There are not more than 4 full weeks in a month. 0 corresponds to the first week, 3 to the fourth. fromBeginningOfMonth is a flag which can be set to TRUE or FALSE. It's easier to understand with example. Thanksgiving is the fourth Thursday of November. In other words it is the fourth thursday from the beginning of the month November. Another example is SysAdmin's day, which is the last Friday of July:

clock_event_initDayOfWeek (FRIDAY, 0, WEEK_FROM_END, JULY, 2000, "SysAdmin's Day")

Day of year

The last type of events is specified by day of year. Example is Programmer's day which is celebrated on the 256 day of a year.

clock_event_initDayOfYear (256, 2009, "Programmer's day")

clock_event_initDayOfYear() is a macro from lib/clock_event.h

// @brief Initializer for an event which is set with day of the year
// @param dayOfYear day of year
// @param year year when the event first occurred
// @param name name of the event
//
#define clock_event_initDayOfYear(dayOfYear, year, name)

Finally

Once you have all your events specified in lib/clock_event_personal.c, don't forget to change CLOCK_EVENTS_SIZE macro in lib/clock_event_personal.h to the size of the event's list. If you have 13 events, CLOCK_EVENTS_SIZE should be 13.

After you have all that ready, you may go to the project root and make the target (compile):

$ cd BinaryClock
$ make

If there is no errors, you may see the result in emulator. From the project root run

$ emulator/build/bin/terminal-binary-clock

Once you're satisfied with result, you can move on and upload the software to the real BinaryClock hardware. Plug FTDI to the BinaryClock and do

$ cd arduino
$ make upload

Saturday, February 1, 2014

BinaryClock - a binary clock DIY

Idea

Here and there on the Internet you may see a geeky device called binary clock. The implementation can be different, but the idea behind is somewhat similar - it is an ordinary digital clock, but showing the time in binary format instead of decimal. After clicking the links you want one for yourself. Here I will tell you what you will need to create one with parts available at almost no cost on eBay.

There are a number of places where you can order a simple device which shows time. My idea was a clock which could show time, date and which could read decimal if I ever want to present that to somebody. Such a device wasn't out there, unfortunately, but looking retrospectively I'm positive it all was for the better because that led me to Arduino.

Arduino is an open-source electronics prototyping platform based on flexible, easy-to-use hardware and software. It's intended for artists, designers, hobbyists and anyone interested in creating interactive objects or environments.

Every Arduino device requires two things - the hardware and the software.

Functionality

The BinaryClock can:

Show time in binary
Slide time in decimal
Show date in binary
Slide date in decimal
Set up time in binary
Set up date in binary
Slide user-defined events from the closest to the furthest
Slide history information about an event such as when an event was first started and how many years the event is being celebrated

To better understand how it works, refer to the next post, where I discuss the hardware. Events are described here.

Software

Most of the time to write a program you don't have to have a device which it is intended to be used on. This code is no exception. Just by spending a little bit more time on design step a developer can get rid of a series of headaches later.

The source tree is organized in following way:

arduino/	Arduino related code
doc/	Documentation and schematic / PCB layout
emulator/	PC emulator
include/	A code which is common for arduino/emulator/lib/test
lib/	BinaryClock library - the clock source
test/	Tests
.ycm_extra_conf.py	I use YouCompleteMe which is a very handy plugin for ViM
LICENSE	The license
Makefile	The Makefile to build BinaryClock
project.vim	A ViM project file - read on :)

The entire source code is published on my GitHub under Apache 2.0 license.

Tools I will need

This is a list of tools you will (or may) need to work with the code. This list will be described in more details further.

Make (comes with every Unix based OS - Linux/Mac, etc.)
Arduino software
Makefile by Sudar / Martin Oldfield
colorgcc (optional, but is very handy when it comes to compiling code in the console/terminal)
Exuberant ctags (optional, but is very handy especially when it comes to Emacs or ViM)

For the emulator and tests:

gcc / clang (at least one of these comes with every Unix based OS - Linux - gcc, MacOs - clang)

For the emulator:

ncurses (comes with every Unix based OS)

For the vimproject (optional - read on):

ViM, MacVim for Mac

For the schematic:

Fritzing

Arduino

Under Arduino folder you will see only two files binary_clock.ino and Makefile.

binary_clock.ino - the code - is less than 200 lines long, half of which is comments. That's because all the functionality is hosted in lib/. This design makes easier testing and reusing the code (see emulator and test below). binary_clock.ino implements the code which is required by Arduino only, such as what pins the display sits on and what pins the buttons are on along with the glue code to bind the BinaryClock to the hardware.

Arduino website popularizes Arduino IDE. While it's ok for playing around, you may find yourself annoyed with the lack of functionality pretty fast. Luckily, there are ways to get around that. I personally like ViM. If you've never used ViM or Emacs - try one of them. Both have a very steep learning curve, but at the end of the day it worth every second. The next thing is Make. It can suit many needs. Just go read about it. To write a good Makefile from scratch is a challenge as the Make system has a very steep learning curve as well. And learning how to do it worth every second as well :). However, in many cases it's just easier to adopt an existing code, than to write your own, especially if it is well supported. And Makefile by Sudar / Martin Oldfield is a very good example of that. I took his Makefile as a basis.

Makefile under arduino folder of the project is definitions for Sudar's Makefile with some extra functionality. It has an error macro in the beginning which checks if Sudar's Makefile is installed. make will throw that error if it is missing.

To compile arduino code, go to arduino folder and run

$ cd BinaryClock/arduino
$ make

To upload the code to a device, run

$ make upload

If you use arduino as ISP programmer, run

$ make ispload

Note: if you don't have colorgcc installed, the Makefile may not work. If it doesn't, just comment two lines of code with "Override Arduino.mk defaults to point to my soft links" above them.

doc

Doc directory contains schematic which comes as a pdf, png and Fritzing project. And as far as the Fritzing project is concerned, there are two parts: FTDI Basic Male Header and Led Matrix LD1088-BS, which you may need to view the project. These parts are not needed for the pdf and the png, though.

emulator

As the name tells, the directory contains emulator related code. Like in the case of Arduino code, emulator is no more than the bindings and the glue code to work with the BinaryClock library (lib/).

Emulator uses ncurses library to emulate the clock in the terminal.

The emulator displays the time the same way it would be displayed on the real piece of hardware. The time follows the standard time representation on any electronic clock - from the left to the right one bar two pixels wide represents hours, minutes and seconds. And the one above reads 12:34:06. BinaryClock uses 24 hour format to eliminate ambiguity. Thus 12 is always 12pm. 12am would have 0 in the first column :)

One more example. On the emulator screen above it reads 23:59:55, which is 11:59pm.

To start the emulator, first you'll have to compile the code. From the project root do

$ make

This will compile everything - the library, Arduino code, the tests and the emulator. If you want to compile emulator only, do this:

$ cd BinaryClock/emulator
$ make

However it will still compile the library code, because it is the core of the BinaryClock. After the compiler has done its job, run

$ emulator/build/bin/terminal-binary-clock

From the project root. The emulator will show up. It will display a welcome message and then it will switch to the time screen. On the right you may notice numbers from 1 to 4 and the text further to the right after each of the numbers. The numbers are buttons. By pressing a corresponding button on the keyboard you execute a behavior associated with the button. On the real device, the buttons are simple push buttons. So, if you press 1, the emulator will show the time in text. BinaryClock has only 8x8 LEDs screen and the text is displayed by sliding it from right to left, the same way it slides on TV during the news hour.

The emulator reads the current time set on your PC when it starts, so you don't have to adjust it. On the real device the time most likely will need to be adjusted.

include

Include folder contains common functionality which is used from all other places. common.h and logger.h are included almost with every *.c unit. Makefile.include is included in the other Makefiles under lib, emulator, test.

lib

lib is a core of the BinaryClock. It is the code which BinaryClock runs. The modules are:

clock	BinaryClock API
clock_alphabet	Characters which BinaryClock supports
clock_button	Functionality to work with the clock buttons
clock_event	Events specific code
clock_event_personal	That's where you may set your own events which the clock will know of
clock_extern	A set of functions which need to be implemented by the user of the library. The pointers need to be assigned with the user-specific implementation
clock_main	The only functionality to initialize and to update the clock state
clock_state	BinaryClock state object - the only object to pass around to the clock
clock_time	A mechanism to update BinaryClock time
date_time	Internal date-time code

The library is compiled into libclock.a which later can be referenced during the compilation of the binaries, such as emulator or test.

test

Contains a bunch of unit tests to test every external function from the lib. That proves that the BinaryClock would behave the way the developer wanted it to behave. test_ut is a light-weight test framework written especially for the project. It futures assert-like functionality which you may be familiar with from the modern test frameworks.

Run

$ cd BinaryClock
$make check

The example output will look like

--- main.c:48 (main) - a test suite started ---
Starting main.c:main --- [ut_clock]
--- ut_clock.c:252 (ut_clock) - a test suite started ---
Starting ut_clock.c:ut_clock --- [clock_slidePattern() returns correct result]
Starting ut_clock.c:ut_clock --- [clock_drawPattern() correct]
Starting ut_clock.c:ut_clock --- [clock_displayBinaryNumber() correct]
Starting ut_clock.c:ut_clock --- [clock_slideText() correct]
--- ut_clock.c:252 (ut_clock) - a test suite completed --- 4 test(s) ran. 4 succeded, 0 failed
Starting main.c:main --- [ut_clock_time]
--- ut_clock_time.c:76 (ut_clock_time) - a test suite started ---
Starting ut_clock_time.c:ut_clock_time --- [clock_updateUptimeMillis() returns correct delta]
Starting ut_clock_time.c:ut_clock_time --- [clock_updateUptimeMillis() handles integers overflows correctly]
--- ut_clock_time.c:76 (ut_clock_time) - a test suite completed --- 2 test(s) ran. 2 succeded, 0 failed
Starting main.c:main --- [ut_date_time]
...
 ...truncated...
...
Starting main.c:main --- [ut_clock_alphabet]
--- ut_clock_alphabet.c:67 (ut_clock_alphabet) - a test suite started ---
Starting ut_clock_alphabet.c:ut_clock_alphabet --- [clock_alphabet_getIndexByCharacter() correct]
Starting ut_clock_alphabet.c:ut_clock_alphabet --- [clock_alphabet_getIndexByCharacter() returns ERANGE and correct index]
--- ut_clock_alphabet.c:67 (ut_clock_alphabet) - a test suite completed --- 2 test(s) ran. 2 succeded, 0 failed
Starting main.c:main --- [ut_clock_event]
--- ut_clock_event.c:330 (ut_clock_event) - a test suite started ---
Starting ut_clock_event.c:ut_clock_event --- [clock_event_initDayOfMonth() is correct]
Starting ut_clock_event.c:ut_clock_event --- [clock_event_initDayOfWeek() is correct]
Starting ut_clock_event.c:ut_clock_event --- [clock_event_initDayOfYear() is correct]
Starting ut_clock_event.c:ut_clock_event --- [clock_event_getEventDetails() is correct]
Starting ut_clock_event.c:ut_clock_event --- [clock_event_initList() is correct]
Starting ut_clock_event.c:ut_clock_event --- [clock_event_updateList() is correct]
Starting ut_clock_event.c:ut_clock_event --- [clock_event_findClosestFromList() is correct]
Starting ut_clock_event.c:ut_clock_event --- [clock_event_yearInfoToStr() is correct]
--- ut_clock_event.c:330 (ut_clock_event) - a test suite completed --- 8 test(s) ran. 8 succeded, 0 failed
--- main.c:48 (main) - a test suite completed --- 6 test(s) ran. 6 succeded, 0 failed

project.vim

This you will only need if you are a ViM geek as I am.

$ alias gvimproject
alias gvimproject='gvim --cmd "source project.vim"'

I have the above alias in my environment and whenever I run gvimproject the shell starts an instance of GViM for me and preloads project.vim from the current folder.

This particular project.vim initializes path variable in ViM essentially telling it where the source code is located. So even if you work with the code under emulator folder and want to do gf (goto file) on a filename which refers to the code in lib, the Vim will know what you are trying to do. It also sets the appropriate ctags for you as you open up a *.c or an *.h file. It rebuilds the appropriate ctags file when you work with the code, so that when you press ctrl+] the tags are always fresh. And finally project.vim saves your current session when you leave Vim and automatically loads the last session when you start Vim. So, you don't have to remember which file was the last one that you worked with, what windows were opened, what line you were at, window size and position, etc.

Monday, September 16, 2013

Strings without escape characters

Does this look familiar?
\"Action\":\"Upload\",
\"FromVersion\":\"ver1\",
\"ToVersion\":\"ver2\"

Wouldn't it be awesome to be able to write like this instead?
"Action":"Upload",
"FromVersion":"ver1",
"ToVersion":"ver2"

Escape characters exists for a simple reason. It says "do not interpret the character after as it should be interpreted". But if we are writing Json we should put escape character way too often. The resulting string could be unreadable. But a simple solution exists:

#define MAKE_STRING(...) #__VA_ARGS__

...

const char *json(MAKE_STRING( {
"Action":"Upload",
"FromVersion":"ver1",
"ToVersion":"ver2" } );

Done! That simple. Here we use a technique called stringification. It accepts a variadic macro, variable number of arguments, in other words. And it stringifies them! Meaning it produces strings. So all the parameters are stringified and then concatenated separately, because two string in C are always concatenated if put next to each other, i.e.

"foo" "bar"

will produce "foobar". Note, that I'm talking about strings here, not the pointers to array of chars or string objects from C++. No, pure strings, such as "I'm a string".

This technique should work on all modern compilers. Was tested on GCC, Clang, MS compiler.

Thursday, September 12, 2013

Exception like stack trace in pure C

All of us have seen stack traces printed out after an exception has been thrown.

Java:

Exception in thread "main" java.lang.NullPointerException
        at com.example.myproject.Book.getTitle(Book.java:16)
        at com.example.myproject.Author.getBookTitles(Author.java:25)
        at com.example.myproject.Bootstrap.main(Bootstrap.java:14)

Python:

Traceback (most recent call last):
  File "test.py", line 8, in b
    c()
  File "test.py", line 13, in c
    assert False
AssertionError

You may find a plenty of information about other languages online.

Stack traces are almost always helpful when debugging. But what if exceptions just don't exist in the language or are forbidden? Well, I don't sympathize C++ exceptions and I don't appreciate exceptions over error codes in other languages, but this is out of the scope of the blog post.

So, let's be more specific. We are in the pure C world. We can't use exceptions. Now what? Do something ugly like:

printf("OMG, something happened\n");
return;

What if we need to finish executing the program if that something happens?

char* ar = malloc(23423423);
if(ar == NULL)
{
printf("oh, ar can't be allocated\n");
exit(errorcode);
}

What if there are a number of functions where ar can be allocated (not the same ar, but just local to function ar)? What if we reuse ar in such a way that it is assigned to a different variable and that malloced again, or realloced? What if we want to easily weed errors out of a normal output? Oh, we could use

fprintf(stderr, "my error");

But what if we now want to change output from stderr to another stream? Or what if we want a uniform format for all of our errors? Or what if we just want to track our error from the origin all the way up to the place where it was called from? Imagine an XML or Json parser which blows up somewhere in a node parsing? What will that error from inside the depths of the parser tell you? Nothing at all, it is generic. It would be much lovelier to see who called that parser and with what arguments. This is where stack trace would be handy. And here is what we are aiming for in C:

Error: example.c:35 (my_function) - [0000001b] custom error
----> someFile.c:50 (super_hero_function) - [0000001b]
----> main.c:163 (main) - [00000001]

This is an example of a proper stack trace. It has an origin in example.c::my_function() and it traces all the way back to main.c::main() through someFile.c::super_hero_function(), but it also outputs a return result of each of those functions on its way.

This is a code which serves this kind of output:

//
// developed by Sergey Markelov (09-2013)
//
#ifndef GENERIC_LOGGER
#define GENERIC_LOGGER
#include <stdio.h>
extern FILE *errStream;
extern FILE *outStream;
#define ERR_STREAM errStream
#define OUT_STREAM outStream
#define ERROR_PREFIX "Error: "
#define STANDARD_PREFIX ""
#define ERROR_CONTINUE_PREFIX " ----> "
#define STRINGIFY(x) #x
#define TOSTRING(x) STRINGIFY(x)
#define _LOG(stream, prefix, format, ...) \
{ fprintf(stream, "%s (%s) - " format "\n", prefix __FILE__ ":" TOSTRING(__LINE__), __func__, ##__VA_ARGS__); }
#define LogError(format, ...) \
{ _LOG(ERR_STREAM, ERROR_PREFIX, format, ##__VA_ARGS__); }
#define Log(format, ...) \
{ _LOG(OUT_STREAM, STANDARD_PREFIX, format, ##__VA_ARGS__); }
//
// @brief this macro originates the error. Functions down the stack should use
// ContinueErrorEx() or ContinueError() to propagate the error behavior.
//
// In the log it will look like:
//
// Error: example.c:35 (my_function) - [0000001b] custom error
// ----> someFile.c:50 (process) - [0000001b]
// ----> main.c:163 (main) - [00000001]
//
// @param result the value which needs to be logged
// @param resultSpecifier printf() specifier for the result. Ex. "%d" means @c result is of type @c int
// you can also use "0x%08x" to make it Hex
// @param format, ... - custom formatted message
//
#define OriginateErrorEx(result, resultSpecifier, format, ...) \
{ LogError("[" resultSpecifier "] " format, result, ##__VA_ARGS__); return result; }
//
// @see OriginateErrorEx
//
#define OriginateError(result, resultSpecifier) \
{ LogError("[" resultSpecifier "] ", result); return result; }
//
// @see OriginateErrorEx
//
#define ContinueErrorEx(result, resultSpecifier, format, ...) \
{ _LOG(ERR_STREAM, ERROR_CONTINUE_PREFIX, \
"[" resultSpecifier "] " format, result, ##__VA_ARGS__); return result; }
//
// @see OriginateErrorEx
//
#define ContinueError(result, resultSpecifier) \
{ _LOG(ERR_STREAM, ERROR_CONTINUE_PREFIX, \
"[" resultSpecifier "] ", result); return result; }
#endif

and here is an example usage:

//
// developed by Sergey Markelov (09-2013)
//
#include "logger.h"
//
// defined in logger.h
//
FILE *errStream;
FILE *outStream;
static int blowUp()
{
if(ftell(NULL) == -1) OriginateErrorEx(-1, "%d", "ftell() caused errno %d - '%s'", errno, strerror(errno));
return 0;
}
int main(void)
{
int res;
errStream = stderr;
outStream = stdout;
res = blowUp();
if(res != 0) ContinueError(res, "%d");
return 0;
}

ftell will fail on NULL file stream and this stack trace will be outputed to stderr:

Error: main.c:15 (blowUp) - [-1] ftell() caused errno [3450] - 'Descriptor is not valid'
----> main.c:27 (main) - [-1]

As you may see we have everything we need to understand what exactly happened and when and who caused that error to happen.

In the code above I used variadic macros. That version is GNU specific and you can't find it in a compiler from MS. Clang supports it out of the box unless you specify to warn about that with -Wgnu. GCC, of course, supports it.

When I say that MS doesn't support variadic macros, that is not to be confused with the full GNU support when you separate format part from the actual variadic part. This way, we can do tricky things like that:

#define _LOG(stream, prefix, format, ...) \
{ fprintf(stream, "%s (%s) - " format "\n", prefix __FILE__ ":" TOSTRING(__LINE__), __func__, ##__VA_ARGS__); }

that is the most generic logging function from the above. It constructs the format string concatenating
"%s (%s) - " with format which is passed as an argument to a macro.

Also I used __LINE__ stringification discussed here.

All that logging mechanism is pure C style macros, thus the original code after preprocessing will be as fast as ordinary fprintf's here and there in your code. So, at the run time we pay no extra cost for a lot of extra functionality!

And all you have to do to use this logging mechanism is to include the h file and to declare errStream and outStream. You can link those to stderr and stdout. Or you can fopen a file and link the stream to there.