Thursday, December 18, 2014

Redmine plus S3 - Secure projects

Prelude

Many developers have been in a position where one way or another they were using a project management tool such as Microsoft TFS, JIRA or others. Some of them are better than others and many of them are paid or require certain components. Many of them are very hardly customizable and many more are not a good fit for a personal project or when the costs should stay low. The best option of all of the low cost alternatives is one which is free. And wide spread. And which has a big community.

Redmine

Redmine is a flexible project management web application. Written using the Ruby on Rails framework, it is cross-platform and cross-database. Redmine is open source and released under the terms of the GNU General Public License v2 (GPL).
Redmine is easy to use, easy to extend and easy to customize software. With a basic skills of CSS it takes very little time to give it a new appearance if you can not find anything suitable out of the plenty themes available for free on the Internet.

Dump

Every developer can recall at least some cases when he has done something and then a catastrophe has happened - he wrongfully ruined his work himself, the software glitch corrupted the data in the database, hardware glitch bricked the hard drive and the information was gone. All of those are examples of a disaster which can hardly be foreseen, but which always can be prevented. Dumping the information regularly will significantly lower if not absolutely remove all consequences of data corruption. Dumps can be local and remote. While local dumps will help you in a situation when the data is corrupted, the remote dumps are essential if the system goes down completely. The best way to dump data is an automated way.

S3

Amazon Simple Storage Service (Amazon S3), provides developers and IT teams with secure, durable, highly-scalable object storage. Amazon S3 is easy to use, with a simple web services interface to store and retrieve any amount of data from anywhere on the web. With Amazon S3, you pay only for the storage you actually use. There is no minimum fee and no setup cost.
S3 is the best option as of the end of 2014 to have you data preserved remotely at low or now cost at all.

Redmine-Dump

The idea is to dump data every day locally and send a weekly backup to S3. The best option is to create a new user specifically for backup reasons with a very restricted permissions.

S3 IAM User policy

From AWS IAM console create an individual IAM user and attach the following policy. You may want to change bucket_name and Version
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::bucket_name/dump/*"
    }
  ]
}
The Resource should be the place on S3 where you want the backups to go. That same resource path should appear in s3-dump.sh S3_DUMP_DIR variable

credentials.dump.sh

credentials.dump.sh is where you define your AWS credentials for the user you've created a few steps above.

$ cat ${HOME}/.config/aws/credentials.dump.sh
export AWS_ACCESS_KEY_ID="A...Q"
export AWS_SECRET_ACCESS_KEY="E...S"

Other things to override

  • common.sh REDMINE_DUMP_DIR - where you want to have your backups at
  • dump.sh REDMINE_HOME - where your Redmine is located
  • s3-dump.sh AWS_REGION - the AWS region you specified when you created an S3 bucket
  • s3-dump.sh S3_DUMP_DIR - where the backups should go to in S3

crontab

Cron is a daemon to have a job to start at a certain day and time.
The corresponding crontab can look like this
$ crontab -l
# 10:00 AM UTC every day - Dump Redmine
0 10 * * * /home/ec2-user/etc/bin/redmine/redmine-dump/dump.sh >> /home/ec2-user/dump/redmine/redmine-dump.log 2>&1

# 11:30 AM UTC every Sunday - Dump Redmine to S3
30 11 * * 0 /home/ec2-user/etc/bin/redmine/redmine-dump/s3-dump.sh >> /home/ec2-user/dump/redmine/redmine-dump.log 2>&1

Friday, August 1, 2014

Браво! downloader

It turned out I like Russian Rock'n'Roll band 'Браво'. You may check it out here: Браво! official website.
The guys made all of the songs available for a free download. The quality is not so good, but hey, it's free.
I also found myself very lazy on clicking each and every link to download all of the tracks. If you feel the same - here is a Python script for your convenience which I've just developed: download.py.
It will download all of the tracks and put them under appropriate folder structure: album/track###.mp3
If it ever stops / fails to fetch a file, you may rerun the script - it is smart enough to understand what files have been downloaded so far, provided that they were not moved and the folder structure hasn't changed.
Enjoy!

Sunday, February 2, 2014

BinaryClock events

In two of my previous posts I was telling about a feature of the BinaryClock which I referred to as events. This is a very important feature which makes BinaryClock personal. It's better to describe it with example.

Example:

Say, today is December 23, 2013. The next event from today would be Christmas which is celebrated on December 25. If you are at the default screen which displays time in binary (refer to the table on this page) and you press button #4 (show event) then you'll see a sliding message "Chrismas - Dec 25 2013 Thursday", now if you press button #2 (show event year info) it will slide "2014 years - started in 0".

Another example:

Your friend Bob is having birthday tomorrow. Today is April 2, 2014. If you press button #4 (show event), you'll see a sliding message "Bob's birthday - Apr 3 2014" if you press button #2 while the event message is sliding, you'll see "25 years - started in 1989" (if Bob's turning 25).

Events can be set in three ways.

Month and day

Example is Christmas which is always celebrated on December 25. To add an event of this type, go to lib/clock_event_personal.c and add
clock_event_initDayOfMonth (25, DECEMBER, 0, "Christmas")
clock_event_initDayOfMonth() is a macro from lib/clock_event.h. Here is its prototype:
// @brief Initializer for an event which is set with month and day
// @param day day of month
// @param month month of the event
// @param year year when the event first occurred
// @param name name of the event
//
#define clock_event_initDayOfMonth(day, month, year, name)

Month and week

Thanksgiving in the US is celebrated on the fourth Thursday of November. To add an event of that type, go to lib/clock_event.personal.c and add
clock_event_initDayOfWeek (THURSDAY, 3, WEEK_FROM_START, NOVEMBER, 1574, "Thanksgiving")
clock_event_initDayOfWeek() is a macro from lib/clock_event.h
// @brief Initializer for an event which is set with
// day of week, week of month, and month
// @param dayOfWeek day of week. See macros in lib/date_time.h
// @param weekOfMonth week of the month. [0..3]
// @param fromBeginningOfMonth whether the _weekOfMonth_ should be counted from the beginning of the month
//          This can be any number or TRUE / FALSE which can be treated in the boolean context
//
// @param month month of the event
// @param year year when the event first occurred
// @param name name of the event
//
#define clock_event_initDayOfWeek(dayOfWeek, weekOfMonth, fromBeginningOfMonth, month, year, name)
weekOfMonth is a number from 0 to 3. There are not more than 4 full weeks in a month. 0 corresponds to the first week, 3 to the fourth. fromBeginningOfMonth is a flag which can be set to TRUE or FALSE. It's easier to understand with example. Thanksgiving is the fourth Thursday of November. In other words it is the fourth thursday from the beginning of the month November. Another example is SysAdmin's day, which is the last Friday of July:
clock_event_initDayOfWeek (FRIDAY, 0, WEEK_FROM_END, JULY, 2000, "SysAdmin's Day")

Day of year

The last type of events is specified by day of year. Example is Programmer's day which is celebrated on the 256 day of a year.
clock_event_initDayOfYear (256, 2009, "Programmer's day")
clock_event_initDayOfYear() is a macro from lib/clock_event.h
// @brief Initializer for an event which is set with day of the year
// @param dayOfYear day of year
// @param year year when the event first occurred
// @param name name of the event
//
#define clock_event_initDayOfYear(dayOfYear, year, name)

Finally

Once you have all your events specified in lib/clock_event_personal.c, don't forget to change CLOCK_EVENTS_SIZE macro in lib/clock_event_personal.h to the size of the event's list. If you have 13 events, CLOCK_EVENTS_SIZE should be 13.

After you have all that ready, you may go to the project root and make the target (compile):
$ cd BinaryClock
$ make
If there is no errors, you may see the result in emulator. From the project root run
$ emulator/build/bin/terminal-binary-clock
Once you're satisfied with result, you can move on and upload the software to the real BinaryClock hardware. Plug FTDI to the BinaryClock and do
$ cd arduino
$ make upload