воскресенье, 11 декабря 2011 г.

Simple logger/debug messages with levels in C (2)

Another version of this task:
#ifndef _DBG_H
#define _DBG_H
#include <stdio.h>
#define PRN(...) printf(__VA_ARGS__)
#define ALIAS_OF_LEVEL(L) (1<<(L))

#ifndef LEVEL
#define LEVEL 0
#endif

#define PRN_L(L, ...) if ((LEVEL) & (L)) PRN(__VA_ARGS__)

#endif

And usage:

#include "dbg.h"

// define levels names (as aliases) and it's number
#define HARDWARE_LEVEL ALIAS_OF_LEVEL(1)
#define WRAPPER_LEVEL ALIAS_OF_LEVEL(2)
#define APP_LEVEL ALIAS_OF_LEVEL(3)

#define LEVEL (HARDWARE_LEVEL|APP_LEVEL)

void main() {
        PRN_L(HARDWARE_LEVEL, "There %d\n", 1);
        PRN_L(WRAPPER_LEVEL, "There %d\n", 2);
        PRN_L(APP_LEVEL, "There %d\n", 3);
}

The difference between first version is that now dbg.h does not limit our levels number. So programmer creates needed levels himself: in the example he created HARDWARE_LEVEL, WRAPPER_LEVEL and APP_LEVEL.

And like in the first version there is not any extra-code generated if PRN_L() is called for disabled level. Look at .s file, generated by gcc for this example:

 .file "dbg.c"
 .def ___main; .scl 2; .type 32; .endef
 .section .rdata,"dr"
LC0:
 .ascii "There %d\12\0"
 .text
.globl _main
 .def _main; .scl 2; .type 32; .endef
_main:
 pushl %ebp
 movl %esp, %ebp
 andl $-16, %esp
 subl $16, %esp
 call ___main
 movl $1, 4(%esp)
 movl $LC0, (%esp)
 call _printf
 movl $3, 4(%esp)
 movl $LC0, (%esp)
 call _printf
 leave
 ret
 .def _printf; .scl 2; .type 32; .endef
And we're seeing only 2 _printf calls - for HARDWARE_LEVEL and APP_LEVEL!

пятница, 9 декабря 2011 г.

Simple logger/debug messages with levels in C

This snippet shows way to turning-on/-off different levels of log/debug messaging in C program:

For example, lets create log messages support with 3 levels and possibility to select what levels are enabled:
#define PRN(...) printf(__VA_ARGS__)
#define L1 1
#define L2 2
#define L3 4

#define PRN_L(L, ...) PRN_##L(__VA_ARGS__)

#if (LEVEL) & (L3)
#  define PRN_L3(...) PRN(__VA_ARGS__)
#else
#  define PRN_L3(...)
#endif

#if (LEVEL) & (L2)
#  define PRN_L2(...) PRN(__VA_ARGS__)
#else
#  define PRN_L2(...)
#endif

#if (LEVEL) & (L1)
#  define PRN_L1(...) PRN(__VA_ARGS__)
#else
#  define PRN_L1(...)
#endif
And somewhere in our program use it:
// enable only messages of levels 1 and 3 (equals to 1|4==5)
#define LEVEL (L1|L3)
#include "logger.h"

void main() {
    // use general macro or PRN_L1(), PRN_L2(), PRN_L3()
    PRN_L(L1, "There %d\n", 1);
    PRN_L(L2, "There %d\n", 2); // no effect!
    PRN_L(L3, "There %d\n", 3);
}
What else? With LEVEL definition you can control enabled levels: 0 - disable all levels or use OR-ed combination of L1..3

среда, 9 ноября 2011 г.

New ypweb version

Added new version of ypweb (Literate Programming tool): 1.1a. This is the bugfix of 1.0a version. See http://code.google.com/p/ypweb/. Plans are to:
  • add options to command (last pie "..; -h")
  • add command line option -h, -h N -- show only N lines of source in doc
  • add command ";; ..." -- means hide other lines of source in doc until next "..."
  • add option to prev. command ";; ...; -l 10" - hide after 10 lines
  • add jumping to tag definition (use ctags) (apart processing doc/src?)
  • add unittests

четверг, 27 октября 2011 г.

LM3S9B92 and FreeRTOS example

Usual all developers starts with example of Web-server on FreeRTOS for this board. But always it does not work: Ethernet link is lost. How to cure it?

First, download last version of FreeRTOS (I tryed FreeRTOSv7.0.2.exe archive). Then unpack it. You can cut unnecessary folders from Demo folder: all except "Common" and "CORTEX_LM3Sxxxx_Eclipse".

Then download Yagarto GNU toolchain (I used yagarto-bu-2.21_gcc-4.6.0-c-c++_nl-1.19.0_gdb-7.2_eabi_20110429) and install it.

Then open main.c in "Demo\CORTEX_LM3Sxxxx_Eclipse\RTOSDemo" and change prvSetupHardware() function to:
void prvSetupHardware(void)
{
 SysCtlClockSet(SYSCTL_SYSDIV_4 | SYSCTL_USE_PLL | SYSCTL_OSC_MAIN | SYSCTL_XTAL_16MHZ);
 SysCtlPeripheralEnable(SYSCTL_PERIPH_GPIOF);
 GPIODirModeSet(GPIO_PORTF_BASE, (GPIO_PIN_2 | GPIO_PIN_3), GPIO_DIR_MODE_HW);
 GPIOPadConfigSet(GPIO_PORTF_BASE, (GPIO_PIN_2 | GPIO_PIN_3 ), GPIO_STRENGTH_2MA, GPIO_PIN_TYPE_STD);
 vParTestInitialise();
}
After it call make in console (Makefile is near main.c :). And now program RTOSDemo.bin into your chip with LM Flash Programmer tool. MAC address of the device should be set. Now reboot MCU and try to ping it - default IP address is 172.25.218.19. You can set static IP of your PC to 172.25.218.10/255.255.0.0 for this reason. And don't forget to disconnect your WiFi :)

There are pong packets! Now try to open in the Web browser http://172.25.218.19 - page will be refresh after 1 second and tasks will be shown.
Thanks to and it's help here.

среда, 26 октября 2011 г.

Hanged LM3S9B92 board

After trying FreeRTOS example for LM3S9B92 my board seems "hanged" (dead) - no response over Ethernet, no adventure game :) And I get message "Error to initializing target".
Last was cured by several turning on/off USB connector. But no game... :) And Finder can not find my board's IP...
After opening Putty on virtual COM port (created after installation of FTDI drivers - I'm talking about Windows:) with baudrate 115200 and reseting board by button on the board, I see message:
Colossal Cave Adventure
-----------------------
Connect to either the USB virtual COM port or
telnet into the Ethernet port in order to play.

MAC Address Not Programmed!
Aha! I ran LM Flash Programmer and on "Other Utilities" tab and:
  1. Reset device by "Debug port unlock" feature (select "Tempest Class", click "Unlock" and follow instructions on message boxes)
  2. Switch to "MAC Address Mode" and programmed MAC address (it's possible to use FF-...-FF).
  3. Re-program device by original firmware (always save original firmwares!)
After this reseting device show in Putty session:
Colossal Cave Adventure
-----------------------
Connect to either the USB virtual COM port or
telnet into the Ethernet port in order to play.

IP: 169.254.3.0
Netmask: 255.255.0.0
Gateway: 0.0.0.0
...after "Wait..." and obtaining IP address. Finder found this IP address, and colossal game is working... over Ethernet :)

вторник, 25 октября 2011 г.

ypweb literate programming tool

After I tried LP technique, I decided that it help me to understand my code better and cuts many possible errors. But I need easier tool, lesser verbose... I implemented very lightweight Python 3 tool (about 20Kb) and called it ypweb. It's hosted on Google.
Main differences with other LP tools are:
  • produces only reStructuredText output
  • very lightweight (only one Python 3 file) about 20Kb
  • doesn't need comment line symbols for document strings (like #!) and code-chunks: only plain text
  • document and code context are switched by special line with short variants
  • generated index contains used by user (not predefined) classifiers
  • doesn't allow code-chunks "placeholders" (defining in one place/inserting in another)
  • supports any input and output charsets (see -h)
Short example:

  1 | Some definitions (macroses) used in COM module:
  2 | ;; x.h; defs; COM definitions
  3 | ;; Port number
  4 | #define PORT 1
  5 | ;; Baudrate
  6 | #define BAUD 9600
  7 | ;; funs; open()
  8 | int open(char *name, char *mode);
  9 | ;;
 10 | Another doc strings...
 11 | ;; x.c; funs; open()
 12 | code of open function here...
 13 | ;; x.h; defs; XXX
 14 | #define XXX
 15 | ;;
 16 | docs strings again...

Line 1 is the document chunk (file may starts with any chunk: doc or code). Line 2 is the special kind of line which switch context to code and begins with special marker (;;) and consists of 3 parts (may be lesser) splited by ;. Both symbols (;; and ;) may be changed to another in command line. It "says": next chunk is code, generates x.h file with some definitions ("defs" but may be any word preferred by user) and is the "COM definitions". These fields will be used also in index generating.

Line 3 is short form of switch-context-line (only caption, no file name or classifier). Line 4 is code and will be saved in the source file. Line 4 is like line 3, next is source line again...

Line 7 is short switch-context-line too and consists of classifier and caption, no filename, so will be saved in the same file - x.h.

Line 9 is empty switch-context-line, so it ends code chunk, nothing else. Next is always doc chunk.
Line 11 is switch-context-line which starts new source file: x.c. Line 13 begins code chunk for early created x.h file. Well, switch-context-line switch into code chunk and defines what is the code if not empty, empty - switches into doc chunk.

It's very clean and simple format. And is the reStructuredText too. So, you can use in it any kind of usual reSt markup!

After processing this file, .rst file and x.h, x.c will be generated. Also .rst file will have index section in the end with references to definitions of the code chunks.

понедельник, 10 октября 2011 г.

AVR timers: calculate prescalers

Functions below are only for Atmega128, see "bitness" of timers, prescalers values...

When you use AVR timers, you should setup it's prescalers' values. Often, programmers calculates prescaler on the paper and then hardcodes it into the code. And this is done every time for new delay value. This task can be done automatically by next algorithm. Next is the Python prototype of the algorithm.
class Timer:
    freq = 0 # freq. of timer
    bits = 0 # "bitsness" of timer

# We have for example 4 timers
_timers = [Timer(), Timer(), Timer(), Timer()]

def init_timers():
    # setup bitness of the timers
    global _timers
    _timers[0].bits = 256
    _timers[1].bits = 65536
    _timers[2].bits = 256
    _timers[3].bits = 65536

# for Atmega128
def _alarm_setup(timer, D):
    prescales0 = (1, 8, 32, 64, 128, 256, 1024,)
    prescales = (1, 8, 64, 256, 1024,)
    # base freq of timer, the same for all but is possible to specify differents
    freq = 8000000 # Freq. of MCU

    if timer == 0:
        psc = prescales0
    else:
        psc = prescales

    _timers[timer].freq = freq

    T = 1000000 * psc[-1] * _timers[timer].bits / freq
    cycles = D/T
    if cycles:
        D -= cycles*T

    for _psc in psc:
        # T - timer period (step in usec)
        T = 1000000 * _psc * _timers[timer].bits / freq
        # D_T - how many T (timer periods) are in D (delay)
        D_T = float(D)/T
        if D_T <= .5:
            # can not be rounded even to 1, too short delay time!
            return None
        elif D_T > _timers[timer].bits:
            # delay is too long for this prescale value
            continue
        else:
            # bingo!
            cn = _timers[timer].bits - int(D_T) + 1
            prescale = _psc
            return (cn, prescale, cycles)
    return None

# test
init_timers()
print _alarm_setup(0, 10000000)
print _alarm_setup(0, 256)

Function _alarm_setup() returns triplet (start value of timer counter, prescaler value, total cycles of the timer counter). These 3 values defines initial parameters of counter which should count to delay D microseconds. It's possible to define any values of delay time in this manner.
This Python prototype example is only to show idea of algorithm. But next is the real implementation on C for WinAVR:
/* Calculates prescale (divisor of timer: 1,8,...) and start
 * counter value and total cycles number. Returns 0 on success, -1 on faulire.
 * D is the delay in microseconds.  timer is number of timer.
 * Out values are saved in corresponding timerc_t.
 * Atmega128 implementation!!!
 */
int
_alarm_setup(int timer,
             unsigned long D)
{
        int i;
        int max_psc; // max. value of prescaler of timer
        int *psc; // prescalers of timer
        unsigned long T; // period of tick
        unsigned long Tclk; // period of clock
        unsigned long Tall; // period of timer cycle (from 0 to bits)

        psc = _timers[timer].prescalers;
        max_psc = psc[_timers[timer].nprescalers-1];

        /* Base frequency is F_CPU. Real frequency is F_CPU/prescaler.
           One tick occupes 1/freq seconds (freq in MHz). N ticks occupes
           N/freq. N is max ticks number in counter = bits of this counter =
           _timers[timer].bits. So, total time to count all ticks of counter is
           N/freq = N/(F_CPU/prescaler) = N*prescaler/F_CPU. Freq is in MHz, to
           use microseconds instead of seconds, * 1000000.

           T is the total time of count all the counter with slowest freq (with
           last prescaler - max_psc). cycles is how many counter cycles are in
           D with slowest prescaler on this timer-counter.

           Then D set to D without cycles time.
           */

        // 1000000 * max_psc > F_CPU, so T/F_CPU != 0
        T = (1000000 * max_psc / F_CPU) * _timers[timer].bits;
        _timers[timer].cycles = D/T;

        if (_timers[timer].cycles > 0) {
                D -= (_timers[timer].cycles) * T;
                _timers[timer].cycles_prescaler = max_psc;
        }
        else {
                // If does not need cycles, cycles_prescaler will be 0
                // (may be used as flag of this)
                _timers[timer].cycles_prescaler = 0;
        }

        D *= 1000; // in nanoseconds
        Tclk = 1000000000 / F_CPU; // in nanoseconds
        for (i=0; i<_timers[timer].nprescalers; i++) {
                T = Tclk * psc[i]; // time of one tick
                Tall = T * _timers[timer].bits; // time to count all ticks of counter
                if (D > Tall) {
                        // delay D is too long for this Tall
                        continue;
                }
                else {
                        _timers[timer].prescaler = psc[i];
                        // D/T how many ticks in D, cn is ticks from 0
                        _timers[timer].cn = _timers[timer].bits - D/T;
                        return (0);
                }
        }
        return (-1);
}

This function uses global array of timers: _timers. This is the array of the timerc_t structs:
typedef struct timerc_t {
        volatile timer_handler_t alarm_handler;
        volatile unsigned long nover; // number of overflow
        unsigned long bits; // number of bits
        volatile unsigned long cn; // start from this count value
        volatile unsigned long cycles; // cycles number
        volatile int prescaler; // prescaler (1, 8, 32... - depends on arch. and timer)
        volatile int cycles_prescaler; // prescaler (usually 1024) for total counter cycles
        int *prescalers; // array of timer-counter prescalers
        int nprescalers; // length of prescalers' array
} timerc_t; 
Full implementation of timers support for AVR (Atmega128 only at the moment) are here.

среда, 21 сентября 2011 г.

Creating of Intel HEX with Tcl/Tk

Intel HEX format is the simple ASCII format usually used to program different devices (it's flash and EEPROM memories). To create such data records, I use this code:
package provide ihex 1.0

namespace eval ihex {
    namespace export data EOF eraser
    set EOF ":00000001FF"

    proc verint {type value} {
        # Verify that value is in the type range. Known types are b (byte),
        # w (word), d (doubleword), q (quadword). If is out-of-range, then
        # raise will be callen
        set absvalue [::tcl::mathfunc::abs $value]

        proc raise {t v} {
            # Raise error message. t is type name, such as "word" or "quadword"
            error [format "Value 0x%X is not a %s" $v $t]
        }

        switch -- $type {
            b { if {$absvalue > 255} {raise byte $absvalue} }
            w { if {$absvalue > 65536} {raise word $absvalue} }
            d { if {$absvalue > 4294967296} {raise doubleword $absvalue} }
            q { if {$absvalue > 18446744073709551616} {raise quadword $absvalue} }
        }
        return 1
    }

    proc args2bytes {args} {
        # Converts args to list of bytes (binary form of args). See
        # >>> args2bytes b1 w4A
        # 01 00 4A
        #
        # Each arg is integer prefixed by b|w|d|q and is in hex-form.
        set bytes [list]
        foreach arg $args {
            set r [scan $arg %1s%x argtype argvalue]
            if {$r != 2} {
                #puts "some string..."
                continue
            } else {
                # verify value of argvalue (in type bounds)
                verint $argtype $argvalue
                # format argvalue into string
                switch -- $argtype {
                    b {set fmt %02X}
                    w {set fmt %04X}
                    d {set fmt %08X}
                    q {set fmt %016X}
                }
                set sarg [format $fmt $argvalue]
                set argbytes [regexp -all -inline .. $sarg]
                set bytes [concat $bytes $argbytes]
            }
        }
        return $bytes
    }

    proc crc {bytes} {
        # Calculate Intel-hex CRC for list of bytes (usual numbers in hex format
        # but without prefixes like b|w|d|q)
        set sum 0
        foreach b $bytes {
            incr sum 0x$b
        }
        set sum [expr {(1 + ~$sum) & 0xFF}] ;# bcb
        return $sum
    }

    proc data {address args} {
        # Returns data record for Intel hex format. args are list of integers such
        # b23 wAE... - byte 0x23, word 0x00AE
        set bytes [args2bytes {*}$args]
        set len [args2bytes [format b%X [llength $bytes]]]
        set address [args2bytes $address]
        set type 00
        set cc [crc [concat {} $len $address $type $bytes]]

        # Prepare output line
        set head :$len[join $address ""]$type
        set tail [join $bytes ""][format %02X $cc]
        return $head$tail
    }

    proc join_eep {args} {
        # Join eep data records (":LLAAAA00..." strings) into one data record with size <= 256.
        # Does not check CRC of input records!
        set MAXSZ 256 ; # DONT USE > 256!!!
        # In the mem will be 256 "00 00 00..."
        for {set i 0} {$i < $MAXSZ} {incr i} { lappend mem 00 }
        set last_index 0
        foreach rec $args {
            set len [string range $rec 1 2]
            set address [string range $rec 3 6]
            set type [string range $rec 7 8]
            if {"0x$type" != 0} { continue } ;# skip not data records
            set bytes [string range $rec 9 end-2]
            set bytes [regexp -all -inline .. $bytes]
            #set ev [eval "expr {0x$address + 0x$len - 1}"]
            set from "0x$address"; set to [expr {"0x$address" + "0x$len" - 1}]
            set last_index [::tcl::mathfunc::max $last_index $to]
            set replacing "lreplace \$mem $from $to $bytes"
            set mem [eval $replacing]
        }
        set len [format b%X [expr {$last_index + 1}]]
        set len [args2bytes $len]
        set address "00 00"
        set type 00
        set mem [lrange $mem 0 $last_index]
        set cc [crc [concat {} $len $address $type $mem]]

        # Prepare output line
        set head :$len[join $address ""]$type
        set tail [join $mem ""][format %02X $cc]
        return $head$tail
    }
}
Procedure ihex::data is for create one record (ASCII line) from it's arguments as:
ihex::data b23 w0A
where each argument is value prefixed by b (for byte), w (for word), d (for doubleword), q (for quadword). Procedure ihex::join_eep is used to "join" into one memory 256-bytes chunk several records. It's arguments are in form of ":LLAAAATTDD...DCC". Procedure args2bytes is not exported but may be usable for creation bytes dump of it's arguments (also used b, w, d, q prefixes).

Changing icons in Windows Tcl Starpacks

Created Starpack will have default Tcl icons and you can see them in different view modes of Windows explorer (table, large icons...). This approach is based on http://wiki.tcl.tk/10922 receipt, but is in Makefile target form. And uses tools: reshacker.exeupx.exe and windres.exe from MinGW installation (at $MINGWHOME/bin/windres.exe), also supposes that you are using tclkit.exe as runtime. In my case I have application rfprg and "virtual" FS (catalog) - rfprg.vfs. And I have all these tools in the same catalog (with tclkit.exe). I created there the rfprg.rc file:
APPICONS ICON "rfprg.vfs/tclkit.ico"
TK ICON "rfprg.vfs/tclkit.ico"
Then I added next target into my Makefile:
icons:
	cp tclkit.exe tclkit1.exe
	upx.exe -d tclkit1.exe
	reshacker.exe -delete tclkit1.exe , tclkit2.exe , icongroup,,
	windres.exe -i rfprg.rc -o rfprg.res -J rc -O res -r
	reshacker.exe -add tclkit2.exe , tclkit3.exe , rfprg.res , ,,
	upx.exe tclkit3.exe
	mv tclkit3.exe tclkit.exe
	rm rfprg.res tclkit2.exe tclkit1.exe
And them:
make icons
This will replace "system" icons of tclkit.exe with rfprg.vfs/tclkit.ico. So you can use this "patched" tclkit.exe runtime to create Starpack with your custom icon.

понедельник, 29 августа 2011 г.

Bottle application helper

Next is the helper class for creation of the Bottle Web Framework applications:
def expose(path, **kw):
    def decor(view):
        view._route_path_ = path
        view._kwargs_ = kw
        return view
    return decor

class BottleBaseApp(object):
    """
    - Autosave attributes with names from __session__ attribute
    - Session type is attr __session_type__ ('cookie'|'pickle'),
      'cookie' is default
    - Methods decorated with @expose will be auto-routed and
      sessions' attrs will be flushed at the end
    """
    def __init__(self, bottleapp):
        self.bottle = bottleapp
        # Prepare session manager as attr sessions
        if self.__session_type__ == "pickle":
            self.sessions = PickleSession()
        else:
            self.sessions = CookieSession()

        def router(meth):
            """Return wrapped view-method"""
            # meth is already bound to self
            def wrp(f=meth, *a, **kw):
                try:
                    ret = f(*a, **kw) # bound, no needs self,
                finally:
                    self.flush_session()
                return ret
            functools.update_wrapper(wrp, meth)
            return wrp

        # Route all decorated methods
        for attrname in dir(self):
            attr = getattr(self, attrname)
            path = getattr(attr, "_route_path_", None)
            kw = getattr(attr, "_kwargs_", {})
            if path:
                self.bottle.route(path, callback=router(attr), **kw)

    def flush_session(self):
        """Copy to session all attrs and save session"""
        try:
            ses = self.sessions.get_session()
            for a in self.__session__:
                try:
                    v = getattr(self, a)
                    ses[a] = v
                except:
                    continue
            self.sessions.save(ses)
        except Exception as x:
            pass
And now is the example how to use this:
class MyApp(BottleBaseApp):
    __session_type__ = "cookie"
    __session__ = ["tasks_ctl_res", "tasks_prc", "prc_conns",]

    def __init__(self, bottleapp):
        super(MyApp, self).__init__(bottleapp)
        self.tasks_ctl_res = {}
        self.tasks_prc = []
        self.prc_conns = {}


    @expose("/index")
    def index(self):
        return "Index page is empty"

    # ... etc ...
I use Session implementation by Sean Reifschneider - MANY THANKS! - (see this).

воскресенье, 21 августа 2011 г.

Internationalization in the Bottle

This approach is very limited! In serious sites you have to support Web-robots, links with language code... I use it for small (embedded Web console) projects only.

I18N in the Bottle Python framework is easy. I prefer to do i18n for 2 different tasks:
  1. i18n for messages (strings)
  2. i18n for static text big pies (in the templates)
I use gettext mechanism. In my application folder I have folders:
locale/
    en/
      TEMPLATES/
        xxx.tpl
    bg/
      LC_MESSAGES/
        messages.mo
      TEMPLATES/
        xxx.tpl
    xx/
      LC_MESSAGES/
        messages.mo
      TEMPLATES/
        xxx.tpl
bg_messages.po
xx_messages.po

"locale" is the folder for i18n/i10n. Folders with language codes ("bg", "xx") are placed here. Each of them contents two subfolders: LC_MESSAGES with compiled .po file (messages.mo) and TEMPLAES (with .tpl files), except en/ folder: no LC_MESSAGES here.

I use make-like mechanism for compilation .po-files, adding new locale - Python fabricate build tool. It's very lightweight and is written in single Python file!

.po-files are very simple. I changed next strings in the one:
"Content-Type: text/plain; charset=utf8\n"
"Content-Transfer-Encoding: utf8\n"
and encoded it into utf-8 (using vim, for example). Then added my msgid, msgstr pairs, as usual.

So, my Bottle application have to know current language (preferred by user) and how to: a) found corresponding template b) how to translate word/text.
# to test, change language in the browser (see priority of them!). Then
# refresh page - you should see text in different language
_LANGS = {}
def initi18n():
    """Init internationalization of UI
    """
    import __builtin__
    global _LANGS
    for l in ("bg","ru"): # list of supported language
        trans = gettext.translation(I18N_DOMAIN, LOCALEDIR, languages=[l])
        _LANGS[l] = trans
    # default translator (i.e. _())
    __builtin__.__dict__["_"] = unicode


def request_lang(req):
    """Determine preferred language of the user from WSGI environment
    variable HTTP_ACCEPT_LANGUAGE. Default is 'en'
    """
    # if cfg module overrides user language
    if cfg.LANG != "default":
        return cfg.LANG

    try:
        return req.environ.get("HTTP_ACCEPT_LANGUAGE", "").split(";")[0].split("-")[0]
    except:
        return "en"


def switch_lang(lang):
    """Select language for output VIA GETTEXT
    """
    import __builtin__
    trans = _LANGS.get(lang, None)
    if trans:
        trans.install(unicode=True)
    else:
        __builtin__.__dict__["_"] = unicode


def templatei18n(name, lang=None, **kw):
    """Like bottle template() but render for current language
    from os.environ["LANGUAGE"]. And not like bottle's template,
    doesn't support source, only name (not path!) as 1st arg!
    If no lang, it will be selected from request
    """
    if not lang:
        lang = request_lang(request)
    tplpath = "%s/%s/TEMPLATES/%s" % (LOCALEDIR, lang, name)
    tplpath = os.path.abspath(tplpath) + ".tpl"
    if not os.path.exists(tplpath):
        tplpath = name
    return template(tplpath, **kw)


def i18n(templ):
    """Decorator for template rendering using current
    language (i18n support)
    """
    def decor(view):
        @functools.wraps(view)
        def w(*a, **kw):
            lang = request_lang(request)
            switch_lang(lang)
            d = view(*a, **kw)
            return templatei18n(templ, lang=lang, **d)
        return w
    return decor

I use initi18n() before application starting and use @i18n decorator for localized templates. Nothing else:
@i18n("tasks") # name of the template
def tasks_list(self):
    """Show table of tasks
    """
    ps = tasks_info()
    for taskname in ps:
        msg = self.runres.get(taskname, "") or ps[taskname]["msg"]
        ps[taskname]["msg"] = msg
        self.runres[taskname] = ""
    return dict(ps=ps)
User should select preferred language in the Web browser options. You - create template "tasks.tpl" in all locale/*/TEMPLATES and translated messages in .mo-file to use _(..) function everywhere you need (Python code, templates).

четверг, 23 июня 2011 г.

Usage of UART for debug messages on AVR

If you need to print out a debug messages to UART port of AVR chip, you can use a simple way for it: switch stdout stream to UART. This is the code of the solution:
#include <avr/io.h>
#include <stdio.h>

#ifndef NDEBUG

#  if !defined(F_DEBUG_UART) && !defined(F_CPU)
#    error F_DEBUG_UART frequency or F_CPU frequency should be defined!
#  endif

#  ifndef F_DEBUG_UART
#    define F_DEBUG_UART F_CPU
#  endif

#  ifndef DEBUG_NUART
#    error DEBUG_NUART should be defined!
#  endif

#  define __CONC3(X, Y, Z) X ## Y ## Z
#  define _CONC3(X, Y, Z) __CONC3(X, Y, Z)
// _CONC3(UDRE, DEBUG_NUART,) does not works
#  define _UDRE (_CONC3(UDR, E, DEBUG_NUART))
#  define _UDR (_CONC3(UDR, DEBUG_NUART,))
#  define _UBRRH (_CONC3(UBRR, DEBUG_NUART, H))
#  define _UBRRL (_CONC3(UBRR, DEBUG_NUART, L))
#  define _UCSRA (_CONC3(UCSR, DEBUG_NUART, A))
#  define _UCSRB (_CONC3(UCSR, DEBUG_NUART, B))
#  define _UCSRC (_CONC3(UCSR, DEBUG_NUART, C))
#  define _RXEN (_CONC3(RXEN, DEBUG_NUART,))
#  define _TXEN (_CONC3(TXEN, DEBUG_NUART,))
#  define _UCSZ0 (_CONC3(UCSZ, DEBUG_NUART, 0))
#  define _U2X (_CONC3(U2X, DEBUG_NUART,))
#  define DEBUG_UART_SUPPORT() \
static int debug_uart_putchar(char c, FILE *stream); \
static FILE mystdout = FDEV_SETUP_STREAM(debug_uart_putchar, NULL, _FDEV_SETUP_WRITE); \
static int debug_uart_putchar(char c, FILE *stream) \
{ \
        while (0 == (_UCSRA & (1<<_UDRE))) ; \
        _UDR = c; \
        return (0); \
}

/* 8 bits data, 1 stop bit, 9600 kbps, async */
#  define INIT_DEBUG_UART() do { \
        stdout = &mystdout; \
        _UCSRC = 0x06; \
        _UCSRA = 0x00; \
        _UCSRB = 0x18; /* receive, transmit, no interrupts */ \
        uint16_t ubrr = (F_DEBUG_UART/16/9600-1); \
        _UBRRL = (unsigned char)ubrr; \
        _UBRRH = (unsigned char)(ubrr>>8); \
} while (0)

#  define DEBUG_UART_PRN(...) printf(__VA_ARGS__)

#else
#  define DEBUG_UART_SUPPORT()
#  define INIT_DEBUG_UART()
#  define DEBUG_UART_PRN(...)
#endif
Save it into debug_uart.h, for example.
What you need to use it, it's only defining F_DEBUG_UART or F_CPU, DEBUG_NUART and calling 3 macros: DEBUG_UART_SUPPORT() on module level (it generates code, so call it only once!), INIT_DEBUG_UART() to init UART and DEBUG_UART_PRN(...) where you need message printing. The example is:
#define F_DEBUG_UART 7372800
#define DEBUG_NUART 0
#include "debug_uart.h"
DEBUG_UART_SUPPORT();
int
main(void)
{
        INIT_DEBUG_UART();
        DEBUG_UART_PRN("Hello world, %d\n", 99);
        ... so on ...
F_DEBUG_UART is the exact amount of the system oscillator frequency to calculate UART baud rate. Default value is equals to F_CPU, but it's possible, that you have value of fosc = 7.3728MHz (see AVR Datasheet about "Examples of UBRR settings") and you use in the Makefile F_CPU value as 8MHz - if you use 8MHz for UBRR calculation too, you will get errors in transmission! So use for F_DEBUG_UART exact values only!
DEBUG_NUART defined number of UART. If you define NDEBUG, all macros will be empty - you don't need to delete it's calls. For more information see help on WinAVR.

вторник, 21 июня 2011 г.

Parsing of paths

Paths are often used in software environments: in file systems, in configuration files, in different IDs and so on. Here are two Python functions, used for parsing of them, they generate tracks from paths (function dothier()) and map based trees (function dottree()). Here you are:
def dothier(seq, sep='.'):
    """Generate path from dot-strings in order for creation
    tree: first - parents then children:
    >>> for x in dothier(["x.y.z", "a"]): print x, x.leaf
    a True
    x False
    x.y False
    x.y.z True

    Every returned string (unicode) has flag - leaf
    """
    # returns this, as unicode but with flag leaf
    _Node = type("dothier_node", (unicode,), {"leaf":False})

    seq = sorted(seq)
    mem = set()
    for p in seq:
        spl = p.split(sep)
        for ip in xrange(1, len(spl) + 1):
            pp = _Node(sep.join(spl[:ip]))
            if len(spl) == 1 or ip == len(spl):
                pp.leaf = True
            if not pp in mem:
                yield pp
                mem.add(pp)

def dottree(seq, sep='.', fullpath=False):
    """Returns dict of dicts like tree from sequence of dot separated strings.
    For example,
    >>> dottree(["x.y.z", "x.y", "a", "a.b.c", "a.b"])
    {'a': {'b': {'c': {}}}, 'x': {'y': {'z': {}}}}

    and

    >>> dottree(["x.y.z", "x.y", "a", "a.b.c", "a.b"], fullpath=True)
    {'a': {'a.b': {'a.b.c': {}}}, 'x': {'x.y': {'x.y.z': {}}}}
    """
    def ins(map, path):
        d = map
        parents = []
        for p in path:
            if fullpath:
                key = sep.join(parents + [p])
            else:
                key = p
            d = d.setdefault(key, {})
            parents.append(p)
    ret = {}
    seq = sorted(seq)
    for p in seq:
        pp = p.split(sep)
        if not pp:
            continue
        ins(ret, pp)
    return ret
Examples of usage are... in it's docstrings :) They are useful in GUI application: to construct tree or another hierarchical controls from paths (retrieved from config. files, for example).
This code is from my project "HMI pybase" (see Projects).

AVR timers support

These modules implements for AVR8, AVR128, AVR168P chips next functions:
void init_timers(void);
int alarm_action(int timer, timer_handler_t alarm_handler);
int _alarm(int timer, unsigned long usec);
int alarm(int timer, unsigned long usec);
unsigned long tclock(int timer);
unsigned long millis(void);
unsigned long micros(void);
alarm() is like standard C function (but for AVR), it's wrapped by cli()/sei() _alarm(). int timer is the number of the used timer. tclock() is like C's clock() but used timer id. clock() is the tclock(0) - standard C function, implemented on timer 0. millis()/micros() returns milliseconds/microseconds from last call. Before use, call init_timer() first.

Before to use, define F_CPU. CLOCKS_PER_SEC should not be defined. If is, this means that some timer for clock() is already in use. Tested with WinAVR-20100110, but may be needs more testing!

You can download source code from here on Google Project page.

пятница, 10 июня 2011 г.

Software SPI implementation for WinAVR

Sometimes you need software SPI MASTER (in my case it was needed to test one chip) implementation. It supports all SPI modes (0, 1, 2, 3), and clock dividers (but so it's software "SPI", that timing is not exact!), but sends only in MSBFIRST bit order (it's easy to add LSBFIRST support).
See the code.
To use it in your project, you need to define some macro-variables (with #define-s) before include the file. They are:
  • SWSPI_MOSI_PORT - port for MOSI "wire"
  • SWSPI_MOSI_PIN - pin for MOSI "wire"
  • SWSPI_MISO_PORT - like above but for MISO (only port)
  • SWSPI_MISO_PIN - line above, but pin for MISO
  • SWSPI_SCLK_PORT - port for SCLK "wire"
  • SWSPI_SCLK_PIN - pin for SCLK "wire"
  • SWSPI_MODE - mode of SPI communication
  • SWSPI_DIV - if not defined, 1 will be used

The example:

#define SWSPI_MOSI_PORT PORTB
#define SWSPI_MOSI_PIN PB3
#define SWSPI_MISO_PORT PINB
#define SWSPI_MISO_PIN PB4
#define SWSPI_SCLK_PORT PORTB
#define SWSPI_SCLK_PIN PB5
#define SWSPI_MODE SPI_MODE0
#define SWSPI_DIV SPI_CLOCK_DIV2

In my real application I included .c file to local file in my project, it's not convenient way for C, but... seems like this:

common/   |  myprj/
swspi.h   |    conf.h (#include <common/swspi.h> and defines macro-vars)
swspi.h   |    spi.c (#include "conf.h" then #include <common/swspi.c>

You can use another scheme, it's trivial.
Here is the code. The .h file:

#ifndef _COMMON_SWSPI_H
#define _COMMON_SWSPI_H

#include <avr/io.h>
#include <util/delay.h>
#include <stdint.h>

#define SPI_CLOCK_DIV4 4
#define SPI_CLOCK_DIV16 16
#define SPI_CLOCK_DIV64 64
#define SPI_CLOCK_DIV128 128
#define SPI_CLOCK_DIV2 2
#define SPI_CLOCK_DIV8 8
#define SPI_CLOCK_DIV32 32
#define SPI_CLOCK_DIV64_2 64

#define SPI_MODE0 0
#define SPI_MODE1 1
#define SPI_MODE2 2
#define SPI_MODE3 3

uint8_t spi_transfer(uint8_t data);
uint8_t spi_ntransfer(const uint8_t *req, int reqlen, uint8_t *resp, int resplen);
#define spi_interrupt_on()
#define spi_interrupt_off()

// for nCS (nSEL) select of device before sending
#define BEGIN_SPI_SENDING(PORT, PIN) do { \
        (PORT) &= ~_BV(PIN); \
        _delay_loop_1(4); \
} while (0)

// for nCS (nSEL) unselect of device after sending
#define END_SPI_SENDING(PORT, PIN) do { \
        _delay_loop_1(4); \
        (PORT) |= _BV(PIN); \
} while (0)

#endif /* !_COMMON_SWSPI_H*/

and the .c file (swspi.c):

#include "swspi.h"
#include <util/delay_basic.h>

#if !defined(SWSPI_MOSI_PORT) || !defined(SWSPI_MOSI_PIN) || \
    !defined(SWSPI_MISO_PORT) || !defined(SWSPI_MISO_PIN) || \
    !defined(SWSPI_SCLK_PORT) || !defined(SWSPI_SCLK_PIN) || \
    !defined(SWSPI_MODE)
#  error Software SPI not configured! SWSPI_* should be configured!
#endif

#define setbit(P, B) ((P) |= (_BV(B)))
#define clibit(P, B) ((P) &= ~(_BV(B)))
#define getbit(P, B) (((P) & (_BV(B)))? 1:0)

#ifdef SWSPI_DIV
#  define SPIDELAYTIME ((SWSPI_DIV)/2)
#else
#  define SPIDELAYTIME 1
#endif
#define SPIHALFDELAY() _delay_loop_1(SPIDELAYTIME) // half of tact (bit transfer) - min 1 CPU tact

#define SETMOSI() setbit(SWSPI_MOSI_PORT, SWSPI_MOSI_PIN)
#define CLIMOSI() clibit(SWSPI_MOSI_PORT, SWSPI_MOSI_PIN)
#define NOMOSI() setbit(SWSPI_MOSI_PORT, SWSPI_MOSI_PIN)

// instead of PORTX may be PINX?
#define READMISO() getbit(SWSPI_MISO_PORT, SWSPI_MISO_PIN)

#if (2 & SWSPI_MODE)
#  define ONSCLK() clibit(SWSPI_SCLK_PORT, SWSPI_SCLK_PIN)
#  define OFFSCLK() setbit(SWSPI_SCLK_PORT, SWSPI_SCLK_PIN)
#else
#  define ONSCLK() setbit(SWSPI_SCLK_PORT, SWSPI_SCLK_PIN)
#  define OFFSCLK() clibit(SWSPI_SCLK_PORT, SWSPI_SCLK_PIN)
#endif

#if (1 & SWSPI_MODE)
#  define SHIFTBIT(outbyte, inbyte) do { \
        (outbyte) & 0x80 ? (SETMOSI()) : (CLIMOSI()); \
        (outbyte) <<= 1; \
        ONSCLK(); \
        SPIHALFDELAY(); \
        (inbyte) <<=1; \
        (inbyte) |= READMISO(); \
        OFFSCLK(); \
        SPIHALFDELAY(); \
} while (0)
#else
#  define SHIFTBIT(outbyte, inbyte) do { \
        (outbyte) & 0x80 ? (SETMOSI()) : (CLIMOSI()); \
        (outbyte) <<= 1; \
        SPIHALFDELAY(); \
        ONSCLK(); \
        SPIHALFDELAY(); \
        (inbyte) <<=1; \
        (inbyte) |= READMISO(); \
        OFFSCLK(); \
} while (0)
#endif

uint8_t
spi_transfer(uint8_t data) {
        int nbit;
        uint8_t res = 0;
        for (nbit=0; nbit<8; nbit++) {
                SHIFTBIT(data, res);
        }
        NOMOSI();
        return (res);
}

/* resp - responce from slave device; resplen - max. expected resp. len (buf. size);
 * if resp is NULL, not saved. Returns always the last byte of the response.
 */
uint8_t
spi_ntransfer(const uint8_t *req, int reqlen, uint8_t *resp, int resplen) {
        int nbit;
        int nbyte;
        register uint8_t outbyte;
        uint8_t inbyte = 0;
        for (nbyte=0; nbyte<reqlen; nbyte++) {
                outbyte = req[nbyte];
                inbyte = 0;
                for (nbit=0; nbit<8; nbit++) {
                        SHIFTBIT(outbyte, inbyte);
                }
                if (resp && nbyte < resplen) {
                        resp[nbyte] = inbyte;
                }
        }
        NOMOSI();
        return (inbyte);
}

#undef ONSCLK
#undef OFFSCLK
#undef SETMOSI
#undef CLIMOSI
#undef NOMOSI
#undef READMISO
#undef SPIHALFDELAY
#undef SPIDELAYTIME
#undef SHIFTBIT
#undef SHIFT0
To send and receive response spi_transfer() and spi_ntransfer() functions are needed. First sends one byte and returns responce's byte (from MISO), seconde - sends several bytes and saves them into 'resp' buffer (contrained by 'resplen') and if resp i NULL, returns the last byte only.
Before any transfer (one or several bytes in the one request) to some chip you should call macro BEGIN_SPI_SENDING(CHIP_NSEL_PORT, CHIP_NSEL_PIN), where CHIP_NSEL_PORT and CHIP_NSEL_PIN defines nCS wire for this concrete chip. After sending you should call END_SPI_SENDING(CHIP_NSEL_PORT, CHIP_NSEL_PIN) to release the wire by nCS.

понедельник, 6 июня 2011 г.

Another approach to Literate Programming

I tried for Literate Programming (LP) two solutions: Nuweb and Python tool Pyweb and found Pyweb tool is better. But with LP there are some problems:
  1. it's not very easy to work with "hidden" source: after tangling http://www.blogger.com/img/blank.gifand weaving you get pure source code and should debug source code (this is usually a "solid" process), after this - should modify .w file

  2. usage of "placeholders" lines split code (difficult for debugging)

Another approach is to use reStructuredText document generator system (see docutils). How to? It's easy: instead of embedding into source code the hyper-text formatting, images, links, you can write your program like story (in usual LP style), use RST syntax (like WiKi) for images embedding and various styles - your code is Literate Programming story as before. But now you can embed code fragments where you need with "include" directive of docutils. This directive requires "startd-after" and "end-before" options for this case. These options defines line (string fragment) the code starts after, and line - the code ends before. I use for this lines Vim foldering lines. See:
// Source code:
int x;
// Opening port {{{
int open_port(int port) {
...
}
// }}}

This is very usuable for folding big source code!
And in your .rst file (which plays a role of the .w file but for weaving only):
and we define the function for a port opening:

.. include:: port.c
:start-after: // Opening port {{{
:end-before: // }}}

Thos technique is not pure LP but is like LP without LP cons. And it's simple documentation approach :)

среда, 1 июня 2011 г.

Programming languages translated to C

There are:
  • ooc
  • BCX (it's Basic)
  • Nimrod
  • Eiffel
  • Scheme (shlep, Gambit-C, Bigloo, Chicken)
  • gcl (it's Common Lisp)
  • Virgil (it's Java-like)
  • Vala, Genie (from GNOME project)
  • Seed7 (Pascal-like)
  • Modula-2/Oberon-2
  • Mercury
and also there are translaters from Algol, Ada, Fortran, Pascal, Ada. Use Google to find it's repos :)

вторник, 31 мая 2011 г.

Multithread queue

This C module is similar to Python Queue - for multithreading programs. It is based on POSIX Threads and BSD queue.h.
  • qitem_t is the queue element (holds void* of data)
  • queue_t is the main struct
  • qerror_t is the enumeration of errors' codes
  • create_queue() is used for creation with maximal elements count limit
  • delete_queue() is the destructor
  • clear_queue() deletes all elements (last arg determines how to freeing elements)
  • put_to_queue() put element
  • get_from_queue() get (and remove) one element if available
  • get_all_from_queue() get all (and remove them) elements from queue

See the code:

#include <assert.h>
#include <stdlib.h>
#include "mtqueue.h"
#ifdef HAVE_LEAKDEBUG
#  include "cMemDbg.h"
#endif
extern void panic(const char *msg);
queue_t*
create_queue(int maxlen) {
        queue_t* q = (queue_t*)malloc(sizeof (queue_t));
        if (!q)
                goto alloc_error;
        SIMPLEQ_INIT(&q->head);
        q->len = 0;
        pthread_mutex_init(&q->qlock, NULL);
        q->maxlen = maxlen;
        return (q);
alloc_error:
    panic("Can not allocate queue");
    return (NULL);
}
void
delete_queue(queue_t *q, void (*delete_data)(void*)) {
        if (q) {
                clear_queue(q, delete_data); // delete all items from queue
                pthread_mutex_destroy(&q->qlock);
                free(q); // free queue
        }
}
void
clear_queue(queue_t *q, void (*delete_data)(void*)) {
        qitem_t *itemp = NULL;
        assert(q);
        if (lock_queue(q)) {
                panic("Can not lock the queue qlock");
                return;
        }
        while (!SIMPLEQ_EMPTY(&q->head)) {
                itemp = SIMPLEQ_FIRST(&q->head);
                if (itemp) {
                        if (delete_data && itemp->data)
                                delete_data(itemp->data);
                        free(itemp);
                }
                SIMPLEQ_REMOVE_HEAD(&q->head, items);
        }
        q->len = 0;
        unlock_queue(q);
}
qerror_t
put_to_queue(queue_t *q, void *data) {
        qerror_t result = (qerror_t)0;
        qitem_t *item = NULL;
        lock_queue(q);
        if (q->len == q->maxlen) {
                result = EQUEUEFULL;
                goto unlock;
        }
        item = (qitem_t*)malloc(sizeof (qitem_t));
        if (!item) {
                result = EQUEUEMEMORY;
                goto unlock;
        }
        item->data = data;
        SIMPLEQ_INSERT_TAIL(&q->head, item, items);
        q->len++;
unlock:
        unlock_queue(q);
        return (result);
}
qerror_t
get_from_queue(queue_t *q, void **dataptr) {
        qerror_t result = (qerror_t)0;
        qitem_t *item = NULL;
        lock_queue(q);
        if (SIMPLEQ_EMPTY(&q->head)) {
                result = EQUEUEEMPTY;
                goto unlock;
        }
        item = SIMPLEQ_FIRST(&q->head);
        *dataptr = item->data;
        SIMPLEQ_REMOVE_HEAD(&q->head, items);
        q->len--;
        free(item);
unlock:
        unlock_queue(q);
        return (result);
}
qerror_t
get_all_from_queue(queue_t *q, void **dataptr, int *num) {
        int i;
        qerror_t result = (qerror_t)0;
        qitem_t *item = NULL;
        void **all = NULL;
        lock_queue(q);
        if (SIMPLEQ_EMPTY(&q->head)) {
                result = EQUEUEEMPTY;
                goto unlock;
        }
        assert(q->len);
        all = malloc(q->len * sizeof (void*));
        if (!all) {
                result = EQUEUEMEMORY;
                panic("Can not allocate result items for get_all_from_queue()");
                goto unlock;
        }
        for (i=0; i<q->len; i++) {
                item = SIMPLEQ_FIRST(&q->head);
                //assert(item);
                if (!item) break;
                all[i] = item->data;
                SIMPLEQ_REMOVE_HEAD(&q->head, items);
                free(item);
        }
        *num = q->len;
        *dataptr = all;
        q->len = 0;
unlock:
        unlock_queue(q);
        return (result);
}

Here is .h file:

#ifndef _MTQUEUE_H
#define _MTQUEUE_H
#include <pthread.h>
#include <queue.h>
typedef struct qitem_t {
        void *data;
        SIMPLEQ_ENTRY(qitem_t) items; /* links */
} qitem_t;
typedef SIMPLEQ_HEAD(qhead_t, qitem_t) qhead_t;
typedef struct queue_t {
        qhead_t head;
        int len;
        int maxlen;
        pthread_mutex_t qlock;
} queue_t;
typedef enum qerror_t {
        EQUEUEFULL = 1,
        EQUEUEEMPTY,
        EQUEUEMEMORY
} qerror_t;
#define FREE_QDATA free
#define lock_queue(q) pthread_mutex_lock(&(q)->qlock)
#define unlock_queue(q) pthread_mutex_unlock(&(q)->qlock)
queue_t* create_queue(int maxlen);
void delete_queue(queue_t *q, void (*delete_data)(void*));
qerror_t put_to_queue(queue_t *q, void *data);
qerror_t get_from_queue(queue_t *q, void **dataptr);
qerror_t get_all_from_queue(queue_t *q, void **dataptr, int *num);
void clear_queue(queue_t *q, void (*delete_data)(void*));
#endif /*!_MTQUEUE_H*/

And simple test:

#include <assert.h>
#include <stdio.h>
#include <stdlib.h>
#include "mtqueue.h"
#include "delays.h"
#define NCICLES 200
queue_t *q = NULL;
void panic(char *msg){
        printf("PANIC: %s\n", msg);
        exit(1);
}
/* Thread of putting an elements to queue.
 */
void* thrput(void *arg) {
        int n;
        int *data;
        int i = *(int*)arg;
        while (i--) {
                printf("put #%d\n", i);
                n = NCICLES;
                while (n--) {
                        data = (int*)malloc(sizeof (int)); /* msg in heap! */
                        if (!data) {
                                panic("Can not allocate data for put");
                        }
                        *data = i*NCICLES + n;
                        while (put_to_queue(q, (void*)data)) {
                            printf("*** ERROR: queue is full!\n");
                            // wait 1 sec and try again
                            pthread_delay(1000);
                        }
                }
        }
}
void thrget1() {
        int i;
        int num = 0;
        void *data = NULL;
        int** idata;
        int prev = -1;
        while (1) {
                data = NULL;
                if (get_all_from_queue(q, &data, &num)) {
                        pthread_delay(2000);
                        if (!q->len)
                                break;
                        else
                                continue;
                }
                printf("get_all_from_queue num=%d\n", num);
                idata = (int**)data;
                for (i=0; i<num; i++) {
                        printf("got %d (prev=%d)\n", *idata[i], prev);
                        assert(prev==-1 || prev-*idata[i]==1);
                        prev = *idata[i];
                        free(idata[i]);
                }
                free(data);
        }
end:
        return;
}

void thrget2() {
        int prev = -1;
        int *data;
        while (1) {
                if (get_from_queue(q, (void**)&data)) {
                        pthread_delay(2000);
                        if (!q>len)
                                break;
                        else
                                continue;
                }
                printf("got %d (prev=%d)\n", *data, prev);
                assert(prev==-1 || prev-*data==1);
                prev = *data;
                free(data);
        }
}
void *thrget(void *arg) {
        thrget2(); // first, get all until NULL
        thrget1(); // then get one by one until NULL
}
void
main() {
        int n = 2;
        pthread_t thrput_id, thrget_id;
        q = create_queue(200); // size of queue
        pthread_create(&thrput_id, NULL, thrput, (void*)&n);
        pthread_create(&thrget_id, NULL, thrget, NULL);
        // wait finishing of the threads
        pthread_join(thrget_id, NULL);
        pthread_join(thrput_id, NULL);
        delete_queue(q, FREE_QDATA);
#ifdef HAVE_LEAKDEBUG
        PrintMemoryLeakInfo();
#endif
}
Tested on Linux, BSD, Windows succesfully.

Validators library

Small but powerfull library of validators for Python. Supports chaining and are based on exceptions raising (it's part of the "HMI Pybase" project, see Projects page):
# -*- coding: cp1251 -*-
from pybase.utils import safe, atof

def _avoid_value(hole):
    """Decorator - return validator which enable any value except
    hole"""
    def f(value):
        if value==hole:
            raise ValueError
        else:
            return value
    return f

class BaseValidator:
    """Base validation. Default behaviour is to `eat` any not None value:

    >>> bv = BaseValidator()
    >>> bv(0)
    0
    >>> bv('a')
    'a'
    >>> bv(None)
    Traceback (most recent call last):
        ...
    ValueError
    """
    def __init__(self, vfunc=_avoid_value(None), hint="", opts={}, **kw):
        """vfunc is validation function which must raise Exception
        (ValueError!) on invalid data and return casted
        value to it's domain. opts is additional options for
        validator, default is value to be returned instead of
        exception raised when is set"""
        self.opts = dict(opts)
        self.hint = hint or self.__class__.__name__
        if "default" in kw:
            self.vfunc = safe(vfunc, kw["default"])
        else:
            self.vfunc = vfunc
    def __call__(self, *a, **kw):
        return self.vfunc(*a, **kw)

class SetValidator(BaseValidator):
    """Validate that value is in set.
    Note, in_ __init__ arg must have __contains__ and __str__.

    >>> sv = SetValidator()
    >>> sv.hint
    'any'
    >>> sv(9)
    9
    >>> sv = SetValidator(in_=(1,2,3))
    >>> sv.hint
    'in (1, 2, 3)'
    >>> sv(3)
    3
    >>> sv(9)
    Traceback (most recent call last):
        ...
    ValueError
    >>> sv = SetValidator("available weight", (1,2,3))
    >>> sv.hint
    'available weight'
    >>> sv(3)
    3
    >>> sv = SetValidator(in_=(1,2), default='unknown')
    >>> sv(0)
    'unknown'
    >>> sv = SetValidator(in_=xrange(0,1000))
    >>> sv(1)
    1
    >>> sv(1)
    1
    """
    def __init__(self, hint=None, in_=None, **kw):
        self.in_ = in_
        hint = hint or ("any" if in_ is None else "in %s"%str(in_))
        def vfunc(value):
            if self.in_ is None or value in self.in_:
                return value
            else:
                raise ValueError
        BaseValidator.__init__(self, vfunc, hint, **kw)

class IntValidator(BaseValidator):
    """Validate that value is integer:
    >>> iv = IntValidator()
    >>> iv("9")
    9
    >>> iv("a")
    Traceback (most recent call last):
        ...
    ValueError: invalid literal for int() with base 10: 'a'
    >>> iv("a", 16)
    10
    """
    def __init__(self, hint=None, **kw):
        BaseValidator.__init__(self, int, hint, **kw)

class FloatValidator(BaseValidator):
    """Validate that value is float:
    >>> fv = FloatValidator("weight", default='no')
    >>> fv.hint
    'weight'
    >>> fv(9)
    9.0
    >>> fv("9.12")
    9.12
    >>> fv("9,12")
    9.12
    >>> fv("a")
    'no'
    """
    def __init__(self, hint=None, **kw):
        def vfunc(value):
            if isinstance(value, str):
                return atof(value)
            else:
                return float(value)
        BaseValidator.__init__(self, vfunc, hint, **kw)

class ChainValidator(BaseValidator):
    """Validators chaining: all validators pipes value. If exception occurs
    in piping then default of this validator will be returned or exception
    occurs (if default is not set):
    >>> sv = SetValidator(in_=(1,2,3,4))
    >>> sv('1')
    Traceback (most recent call last):
        ...
    ValueError
    >>> iv = IntValidator(default=4)
    >>> av = ChainValidator(and_=(iv, sv), default='what?')
    >>> av('1')
    1
    >>> av('2')
    2
    >>> av('10')
    'what?'
    >>> av('x')
    4
    >>> av.hint
    'IntValidator & in (1, 2, 3, 4)'
    >>> av = ChainValidator(and_=(iv, sv), hint="int list", opts={"something":123})
    >>> av.hint
    'int list'
    >>> av.opts["something"]
    123
    """
    def __init__(self, and_, hint=None, **kw):
        self.and_ = and_
        def vfunc(value):
            v = value
            for validator in and_:
                v = validator(v)
            return v
        if not hint:
            h = []
            for v in and_:
                h.append(str(v.hint))
            h = " & ".join(h)
            #if h: h = "[%s]"%h
        else:
            h = hint
        BaseValidator.__init__(self, vfunc, h, **kw)

if __name__=="__main__":
    import doctest
    doctest.testmod()

Here are examples of usage (and extending):
def IntSetValidator(hint, in_, *a, **kw):
    """Checks does int set contents a value"""
    iv = IntValidator()
    sv = SetValidator(in_, *a, **kw)
    return ChainValidator(hint=hint, and_=(iv, sv))

class IntRangeValidator(BaseValidator):
    "Checks does value is contained in 'a-b' range"
    def __init__(self, hint=None, **kw):
        def vfunc(value):
            """Should raise ValueError (ONLY) on invalid value, else
            returns parsed value"""
            try:
                rng = [int(x.strip()) for x in value.split('-')]
                if rng[1] <= rng[0]:
                    raise Exception
                else:
                    return rng
            except:
                raise ValueError
        BaseValidator.__init__(self, vfunc, hint, **kw)

TinyBSD installation

This is:
  • Live USB system
  • Installer of TinyBSD (MFS based FreeBSD 8) on 16Mb (of system partition) for embedded devices
(see http://code.google.com/p/tinybsd-inst/ for downloads).
Script 'build.my' is used for image creation and copy to USB flash (all information will be destroyed!). Image is 32Mb. Original 'tinybsd' script is modified to support lists of includes and excludes files from image (see 'tinybsd.patch') via 'tinybsd.nofiles', 'tinybsd.basefiles' (last was in original TinyBSD).
'kh_inst' is the project/conf (similar to 'firewall', 'vpn', so on). Installation is made throught rc.d mechanism: etc/rc.d/osinst script will be called at start time and will ask user to boot from USB with live FS or to install system to HDD.
/etc/instskel contents all configuration of installed system:
  • bsdlabel
  • csh.cshrc
  • fdisk
  • fstab
  • motd
  • nodes
  • profile
  • rc.conf
  • ttys
...all are standard except 'nodes'. See them to know how installer will create slices, partitions, FSs, run-time configs... As to 'nodes', it's only predefined values of IP addresses of predefined nodes, user can select IP by node's name (when install system) or to enter the value manually.
Installed OS additionally contents 2 preinstalled tools: mksh shell (small but usuable from MirBSD project) and tmux - BSD console multiplexer.

This installer was tested on LX-800 (AMD based board for embedded solutions).

понедельник, 30 мая 2011 г.

Skeleton files for VIM

Very simple script for auto-create of file content by it's extension:
" Use :Skel to apply skeleton (template) to empty new file (skeleton is
" dot-file from vimfiles/skel/dot.
function _Skel()
    " for any files
    norm 1G0
    let ext = expand('%:e')
    let dotfile = $VIM . '/vimfiles/skel/dot.' . ext
    exec 'r ' . dotfile
    norm 1Gdd
endfunction
function HSkel()
    " for .h files
    call _Skel()
    let HGUARG = toupper(expand('%:t:r'))
    exec 'silent %s/%FILE%/' . HGUARG . '/g'
    norm G-
endfunction
function Skel()
    let ext = expand('%:e')
    if ext == 'h'
        call HSkel()
    elseif ext == 'w'
        call _Skel()
    endif
endfunction
command! -buffer Skel :call Skel()

For example, I have in vimfiles/skel/dot.h:
/*
 * $Id$
 */

#ifndef _%FILE%_H
#define _%FILE%_H


#endif /* !_%FILE%_H*/

and after creation of some.h I call :Skel and get:
/*
 * $Id$
 */

#ifndef _SOME_H
#define _SOME_H


#endif /* !_SOME_H*/

Animated plot with matplotlib

This is the components of auto-scrolled plots with matplotlib:
class Plot:
    """Base attributes of an any plot"""
    def __init__(self,
            fig=None, # matplotlib Figure
            canvas=None, # matplotlib Canvas
            tkw=None, # Tk widget
            title="", #
            xdata=[], # x data
            ydata=[], # y data
            xmin=None, # left xlim:datetime (chages in scroll with set_xlim)
            xmax=None, # right xlim:datetime (chages in scroll with set_xlim)
            tdelta=None, # step (period of polling of sensors, etc.):timedelta
            ylabel="", # y label
            lnc="red", # line color
            lnpt="-", # line point type
            lnw=1, # line width
            intc="#707070", # internal rect. color
            outc="#9E9E9E", # outter rect. color
            ylimpad=10, # padding around y axis limit (ylim is +/- ylimpad)
            ):
        self.fig = fig
        self.canvas = canvas
        self.tkw = tkw
        self.title = title
        self.xdata = xdata
        self.ydata = ydata
        self.xmin = xmin
        self.xmax = xmax
        self.tdelta = tdelta
        self.xband = self.xmax - self.xmin # width of axe 0x (timedelta)
        self.ylabel = ylabel
        self.lnc = lnc
        self.lnpt = lnpt
        self.lnw = lnw
        self.intc = intc
        self.outc = outc
        self.ylimpad = ylimpad

        self.ymin = 0
        self.ymax = 0

class AnimPlot(Frame):
    """One-line plot in own frame"""
    # TODO: blit not used
    def __init__(self, master, app, blit=False, title="", npoints=60, ylabel="", lnc="red", lnpt="-", lnw=1, intc="#707070", outc="#9E9E9E", ylimpad=10, period=1, *a, **kw):
        Frame.__init__(self, master, *a, **kw)
        self.app = app

        xmin = self.app.begin
        xmax = xmin + timedelta(seconds=npoints*period) # period is secs
        tdelta = timedelta(seconds=period)

        self.p = Plot(title=title, ylabel=ylabel, lnc=lnc, lnpt=lnpt, lnw=lnw, intc=intc, outc=outc, xdata=deque(maxlen=npoints), ydata=deque(maxlen=npoints), xmin=xmin, xmax=xmax, tdelta=tdelta, ylimpad=ylimpad)
        self.blit = blit # use double-buffering?
        self._create_matplot()
        self.p.tkw.pack()

    def _create_matplot(self):
        """Creates of matplotlib plot and changes
        self.p"""

        # calculate plot size in inches; dpi is the pixels count per inch
        dpi = 75.
        figh = self.winfo_screenheight()
        figh = figh/dpi - 1 # 1 for menu bar
        figh = figh/2 # two plots
        figw = figh*2.5 # aspect
        fig = Figure(figsize=(figw, figh), dpi=dpi)

        fig.patch.set_facecolor(self.p.outc)
        ax = fig.add_subplot(111)
        ax.patch.set_facecolor(self.p.intc)
        ax.set_title(self.p.title)
        ax.grid(True)
        ax.set_ylabel(self.p.ylabel)

        ax.plot_date([], [], self.p.lnpt, color=self.p.lnc, lw=self.p.lnw)
        ax.set_xlim(mdates.date2num(self.p.xmin), mdates.date2num(self.p.xmax))

        ax.xaxis.set_major_formatter(mdates.DateFormatter('%H:%M:%S', utils.tzinfo()))
        ax.fmt_xdata = mdates.DateFormatter('%H:%M:%S', utils.tzinfo())

        canvas = FigureCanvas(fig, master=self)
        tkw = canvas.get_tk_widget()
        tkw.config(relief=FLAT, bd=0, takefocus=0, highlightthickness=0)
        self.p.fig = fig
        self.p.canvas = canvas
        self.p.tkw = tkw

    def _redraw(self):
        """Redraws plot with data from self.p automatically after append of the new data"""
        ax = self.p.fig.axes[0]
        line = ax.lines[0]

        line.set_xdata(self.p.xdata)
        line.set_ydata(self.p.ydata)

        x1 = self.p.xdata[-1]
        tm0 = mdates.num2date(x1) - self.p.xband + self.p.tdelta
        x0 = mdates.date2num(tm0)
        ax.set_xlim(x0, x1)

        y0 = 0
        y1 = self.p.ymax
        ax.set_ylim(y0 - self.p.ylimpad, y1 + self.p.ylimpad)
        self.p.canvas.draw()

    def appendy(self, y):
        tm = utils.now()
        tm = mdates.date2num(tm)
        self.p.xdata.append(tm)
        self.p.ymax = max(self.p.ymax, y)
        self.p.ydata.append(y)
        self._redraw()
Here is an example of usage:
aplt1 = AnimPlot(self.pltfr, app=self.app, name="tpm200plot1", lnc="red", lnw=2, title=u"Sensor PV1", ylabel=u"C", period=3)
aplt1.pack(side=TOP, expand=YES, fill=Y)
# ...
def onpolling():
    val = read_sensor()
    aplt1.appendy(pv1)

Crossplatform software delays

This is the simple macros for different delays usuable on Unix and Windows:
#ifndef _DELAYS_H
#define _DELAYS_H

#include 
#include 

/* Converting between seconds and CPU ticks (clock_t):
 *      MS2TICKS() converts milliseconds to ticks
 *      US2TICKS() converts microseconds to ticks
 *      TICKS2MS() converts ticks to milliseconds
 *      TICKS2US() converts ticks to microseconds
 */
#define MS2TICKS(MS) ((MS)*(CLOCKS_PER_SEC)/1000)
#define US2TICKS(US) ((US)*(CLOCKS_PER_SEC)/1000000)
#define TICKS2MS(T)  ((T)/((CLOCKS_PER_SEC)*1000))
#define TICKS2US(T)  ((T)/((CLOCKS_PER_SEC)*1000000))

/*
 * delay(MS)         sleeping on MS milliseconds
 */
#if defined(WIN32) || defined(_WIN32) || defined(__WIN32)
#  include 
#  define delay(MS) Sleep(MS)
#else
#  if defined(HAVE_USLEEP)
#    define delay(MS) usleep((MS)*1000)
#  elif defined(HAVE_SLEEP)
#    include 
#    define delay(MS) sleep((MS)/1000) /* granularity only ONE SECOND! */
#  else
#    error Can not found any sleep_clocks() implementation ideas
# endif
#endif

/*
 * pthread_delay(MS) sleeping thread on MS milliseconds
 */
#if defined(HAVE_PTHREAD_DELAY_NP)
#  define pthread_delay(MS) \
     do { \
             struct timespec __timespec={(MS)/1000,((MS)%1000)*1000000000}; \
             pthread_delay_np(&__timespec); \
     } \
     while(0)
#else
#  define pthread_delay(MS) delay(MS)
#endif

/*
 * unow(PNOW) store in var pointed by PNOW microseconds since Epoch
 */
#define unow(PNOW) \
{ \
        struct timeval __tv;  \
        gettimeofday(&__tv, NULL); \
        *(PNOW) = (__tv.tv_sec * 1000000) + __tv.tv_usec; \
}

#endif /*!_DELAYS_H*/
If HAVE_PTHREAD_DELAY_NP will be defined before including of this file, then pthread_delay() will be wrapper for pthread_delay_np() - see Pthreads for Win32 headers.

суббота, 28 мая 2011 г.

Bakefile: Makefile's generator

It's known problem - general way of making the project. There are many tools for this task: CMake, automake/autoconf, ant (usual for Java) and many-many other. But some of them use own build system and very few uses standard way of Makefile (i.e. generates Makefiles). This method is based on some input format which is translated to Makefile for target compiler. For example, automake/autoconf, Premake (with Lua scripting) and Bakefile (written in Python and is crossplatform, like Premake and many other). It's authors are Vaclav Slavik, Francesco Montorsi.

Bakefile uses XML as input format then generates standard Makefiles for the targets:

autoconf       Makefile.in for GNU Autoconf
borland        Makefile for Borland C++ and Borland make
dmars          Generic Makefile for Digital Mars C/C++
dmars_smake    Makefile for Digital Mars C/C++ with SMAKE
gnu            Makefile for GNU toolchain: GNU Make, GCC...
mingw          Makefile for MinGW: mingw32-make, MinGW...
msvc           Makefile for Visual C++ with Microsoft nmake
msvc6prj       Microsoft Visual C++ 6.0 project files
msevc4prj      Microsoft Embedded Visual C++ 4 project files
msvs2003prj    MS Visual Studio 2003 project files
msvs2005prj    MS Visual Studio 2005 project files
msvs2008prj    MS Visual Studio 2008 project files
suncc          GNU makefile for SunCC compiler
symbian        Symbian development files
watcom         Makefile for OpenWatcom C/C++
xcode2         Apple Xcode 2.4 project files

It's input file looks like:
<exe id="test_sockets">
        <cflags>-fpack-struct</cflags>
        <sources>test_sockets.c</sources>
        <sources>socks.c</sources>
        <sys-lib>pthread</sys-lib>
        <if cond="TOOLSET=='win32'">
            <sys-lib>wsock32</sys-lib>
        </if>
</exe>

This pie of code will build (via native Makefile sure) test_sockets binary from test_sockets.c and socks.c, with pthread library (available on Unix and MinGW too) and with wsock32 library only on Windows.

Another example of custom tag:
<include file="presets/simple.bkl">
    <define-tag name="lit" rules="action">
        <if cond="TOOLSET=='unix'">
            <command></command>pyweb.py -s $(attributes['web']).w
            <command></command>rst2html.py $(attributes['web']).rst $(attributes['web']).html
        </if>
        <if cond="TOOLSET=='win32'">
            <command></command>cmd /C pyweb.py -s $(attributes['web']).w
            <command></command>cmd /C rst2html.py $(attributes['web']).rst $(attributes['web']).html
        </if>
    </define-tag>

    <action id="lit">
        <lit web="maintest">
        <lit web="socks">
    </lit></lit></action>
</include>

This snippet will weave and tangle "literated" sources (with pyweb tool): maintest.w and socks.w but in different manners under Windows and Unix.
All of these snippets should be in <makefile> tag! When you use some tag, see documentation on tag (what outer tags it expect).

Calling Gimp via shell

This Python class is a wrapper for Gimp and use shell to communicate. It's part of the "HMI pybase" project, see on Project's page. It supports batch mode, rc file (config), session cleaning. Example of usage:
def gen_gimprc(gfig_dir):
    """Contents of rc file: setup directory of GFig plugin files
    """
    ret = u"(gfig-path \"%s\")"%gfid_dir.replace('\\', '\\\\') # should be (gfig-path "xx\\xx")
    return ret

...

b = Gimpsh(gimp_exe_path, locals())
gimprc = gen_gimprc(gfid_dir)
b.append((u"'(sym-new \"%(fpath)s.%(fmt)s\" %(indexed)d %(width)d %(height)d)'",), gimprc=gimprc)
try:
    # call some Scheme script sym-new with pre-setup rc file
    b.execall()
except Exception, x:
    print "Error!"
Here are the code:
import tempfile
import sys
import os
from pybase import utils

class Session(list):
    """list of jobs that ran on the same Gimp instance
    """
#    def __unicode__(self):
#        s = u"%s (%s)"%(super(Session, self).__str__(), self._opts)
#        return s

    _tmpfiles = [] # paths of temp. files - STATIC!
    def __init__(self, *a):
        super(Session, self).__init__(*a)
        self._opts = []
        self._args = []

    def __del__(self):
        # remove all temporary files
        for p in Session._tmpfiles:
            try:
                os.unlink(p)
            except:
                pass

    def copy(self):
        """Independed copy"""
        items = self[:]
        cpy = Session(items)
        cpy._opts = self._opts
        cpy._args = self._args
        return cpy

    def set_opt(self, opt, value=None):
        """option opt (-x|--x), value may be used or not. In _opts will be
        '-x'|'-x xxx'|'--x'|'--x=xxx'
        """
        if value and sys.platform=="win32":
            value = utils.quote_win32(value)
        if opt.startswith("--"):
            # long option
            f = u"%s=%s"%(opt,value) if value else opt
        else:
            f = u"%s %s"%(opt,value) if value else opt
        self._opts.append(f)

    @property
    def gimprc(self):
        raise AttributeError
    @gimprc.setter
    def gimprc(self, value):
        """Exception raising is possible
        """
        if not value: return
        (fd, name) = tempfile.mkstemp(prefix="gimprc", text=True)
        Session._tmpfiles.append(name)
        with os.fdopen(fd, "w") as f:
            f.write(value.encode("utf8"))
            f.close()
            self.set_opt("-g", unicode(name))

    def set_args(self, value):
        """Exception raising is possible
        """
        if not value: return

        if sys.platform=="win32":
            value = [utils.quote_win32(v) for v in value]
        self._args = value

    @property
    def cmdline(self):
        return u"%s %s"%(u" ".join(self._opts), u" ".join(self._args))

class Gimpsh:
    """Call gimp via shell. Called command lines are appened
    by append(). It's one call or many with several instances.
    It uses jobs (runnable via -b .. mechanism), so several
    Scheme functions can be called on one Gimp instance
    """
    def __init__(self, gimp="gimp", vars=None):
        """gimp is the full path to gimp program.
        Vars is the dict of variables for formatting, can be
        changed on the fly.
        """
        self.gimp = gimp
        # list of lists: each list is ran in one task, last task is
        # interactive, other are not.
        self._bs = []
        self.vars = vars or {}

    def append(self, jobs=None, args=None, gimprc=None):
        """Append list of jobs, returns job-sequence id.
        kw are the sessions settings, like gimprc path.
        args are list of positional arguments.
        """
        if not jobs:
            jobs = []
        if type(jobs) not in (list, tuple):
            raise ValueError("jobs should be sequence")
        ret = len(self._bs)

        # create Session with attributes (for options)
        ses = Session(jobs) # list of jobs with session settings
        ses.gimprc = gimprc
        ses.set_args(args)

        self._bs.append(ses)
        return ret

    def _shsession(self, jobs, last):
        """Execute only jobs of one sequence on one gimp instance (one
        session), last is the opt of last sequence (last item in self._bs!)
        """
        if len(jobs) != 0:
            if not last:
                jobs = jobs.copy()
                # FIXME but final console message is not close, so until console is close
                # starting of next gimp is impossible, may be workaround with another
                # system() without waiting child PID
                jobs.append(u"'(gimp-quit 0)'")
                jobs.set_opt("-i")
            j = Gimpsh._prebatch(jobs, self.vars) # returns real list (not Session) of -b OPTs
        else:
            j = ""
        #jobs.set_opt("-s") # no splash window
        cl = u"%s %s %s" % (self.gimp, j, jobs.cmdline)
        utils.system(cl)

    def execone(self, jobsid):
        """Like _shsession but executed jobs are pointed by jobsid. If jobsid
        out of range, IndexError will be raised
        """
        self._shsession(self._bs[jobsid], jobsid==len(self._bs)-1)

    def execall(self, vars={}):
        """Executes all jobs sequence
        """
        for jobsid in xrange(len(self._bs)):
            self.execone(jobsid)

    @staticmethod
    def _prebatch(batches, vars={}):
        """Prepare batch for command line: quotes on win32, adds -b
        """
        if sys.platform ==  "win32":
            bs = [u"-b "+utils.quote_win32(b%vars) for b in batches]
        else:
            bs = [u"-b "+b%vars for b in batches]
        return u" ".join(bs)

Virtual root (Python module)

Virtual root is for sandbox - portable file system pie. May be used when app need portable FS pie (see "vroot" module in "HMI pybase" on Projects page):

VRoot:
  usr/
    share/
  lib
  ...etc...

And app have to access files under relative paths in VRoot (sandbox).

VRoot(mount_point) create virtual (sandbox) root with methods:
  • hpath(virtual_path) returns path on host
  • vpath(host_path, keep_host=False) returns virtual path, if not possible and keep_host, returns original host_path (None otherwise)
And function:
  • path2uri(path, base=None) returns URI, like a/b/c to be inserted to 'file:///HERE'

from os import path

class VRoot:
    """Use always unicode in path!
    """
    _mp = None

    def __init__(self, mp):
        "mp is the mount-point: abs. path on host"
        self.mount(mp)

    def hpath(self, vp):
        """Returns host path from virtual path:
        >>> vr = VRoot(u'/a/b')
        >>> vr.hpath(u'c///') == VRoot._fixed_abspath(u'/a/b/c')
        True
        >>> vr.hpath(u'/c///') == VRoot._fixed_abspath(u'/a/b/c')
        True
        >>> vr.hpath(u"../../../c/") == VRoot._fixed_abspath(u'/a/b/c')
        True
        >>> vr.hpath(u"/../c/") == VRoot._fixed_abspath(u'/a/b/c')
        True
        """
        vp = vp.replace(u"\\", u"/").split(u'/')
        vp = u'/'.join(x for x in vp if x and x!=u"..")
        return VRoot._fixed_abspath(path.join(self._mp, vp))

    def vpath(self, hp, keep_host=False):
        """Returns virtual path, relative to _mp. If keep_host, then
        returns original hp when hp is not in _mp, otherwise returns None:
        >>> vr = VRoot(u'/a/b/')
        >>> vr.vpath(u'/a/b/') == path.sep
        True
        >>> vr.vpath(u'/a/b/', True) == path.sep
        True
        >>> vr.vpath(u"/x///")
        >>> vr.vpath(u"/x/", True) == path.sep + u'x'
        True
        >>> vr = VRoot(u'/')
        >>> vr.vpath(u'/a/b/') == path.normpath(u"a/b")
        True
        """
        ap = VRoot._fixed_abspath(hp)
        ap = VRoot._fixed_relpath(ap, self._mp)
        if u".." in ap:
            if keep_host:
                return path.normpath(hp)
            else:
                return None
        elif ap == u".":
            return path.sep
        else:
            return ap

    @staticmethod
    def _pathroot(p):
        """Returns absolute path of root of path p:
        >>> VRoot._pathroot("/a/") == VRoot._fixed_abspath("/")
        True
        >>> VRoot._pathroot(r"d:\\x\\y\\\\")
        'd:\\\\'
        """
        p_drive = path.splitdrive(p)[0]
        if p_drive:
            # if there is the drive, absroot will be ''
            absroot = p_drive + path.sep
        else:
            # else will be default system root
            absroot = VRoot._fixed_abspath(u"/")
        return absroot

    @staticmethod
    def _fixed_abspath(p):
        """Issue: not normalized case"""
        return path.normcase(path.abspath(p))

    @staticmethod
    def _fixed_relpath(p, base):
        """Issue: when base is root, p is prefixed by '..':
        >>> VRoot._fixed_relpath('/a/b/c/', '/') == r'a\\b\\c'
        True
        >>> VRoot._fixed_relpath(r'd:\\x\\y', r'd:\\\\') == path.normpath("x/y")
        True
        """
        base_drive = path.splitdrive(base)[0]
        if base_drive:
            # if there is the drive, absroot will be ''
            absroot = base_drive + path.sep
        else:
            # else will be default system root
            absroot = VRoot._fixed_abspath(u"/")
        if VRoot._fixed_abspath(base) == absroot:
            # if base is root
            return path.normpath(p).lstrip(u".." + absroot)
        else:
            return path.relpath(p, base)

    def mount(self, mp):
        self._mp = VRoot._fixed_abspath(mp)

def path2uri(p, base=None):
    """Path to uri, but without 'file:///'! See:
    >>> path2uri('/a/b/c///')
    u'a/b/c'
    >>> path2uri(r'd:\\x\\y')
    u'x/y'
    >>> path2uri(u'/x/y/z', '/x')
    u'y/z'
    """
    return VRoot(base or VRoot._pathroot(p)).vpath(p).replace('\\', '/')

if __name__ == "__main__":
    import doctest
    doctest.testmod()
    #print VRoot._pathroot(r'd:\\\\x\y')

Multithreading console

There are several Unix programes with this feature: user can edit input line since program raise messages to console:
Message 3
Message 2
Message 1
enter name$ Alejandr_
It's implementation for Windows is simple (see Module "mtinput" in HMI pybase project on Projects page):
# Multi-thread console input() via class MTInput
import sys
import msvcrt
import string
from threading import Lock

# FIXME Needs Win32 module msvcrt! Under Linux is possible to use
# tty (but not on FAT FS system like in Linpac controllers!)
PROMPT = ">>>"
def normstr(s):
    """Interpret backspaces in string s,
    returns normal string
    """
    rs = reversed(s)
    res = []
    skip = 0
    for ch in rs:
        if ord(ch) == 8:
            skip += 1
            continue
        elif skip:
            skip -= 1
            continue
        else:
            res.append(ch)
    return u"".join(reversed(res))

class MTInput:
    """Multi-thread input line. Can be used like:

    import time
    import thread

    inputer = MTInput()
    def f():
    while 1:
        print '\ntest.......'
        inputer.update()
        time.sleep(2)
    thread.start_new_thread(f, ())
    while 1:
        inp = inputer.input()
        print 'ENTERED:', inp
        if inp == ".":
            break
        else:
            # ... process ...
    """
    def __init__(self, stdout=None, prompt=PROMPT):
        self._stdout = stdout or sys.stdout
        self._enc = getattr(self._stdout, "encoding", "ascii")
        self._prompt = prompt
        self._buf = []
        self._lock = Lock()

    def update(self):
        with self._lock:
            buf = u"".join(self._buf)
            try:
                s = u"%s %s" % (self._prompt, buf)
                encs = s.encode(self._enc)
                self._stdout.write(encs)
                self._stdout.flush()
            except:
                pass

    def input(self, prompt=PROMPT):
        with self._lock:
            self._buf = []
            self._prompt = prompt
        self._stdout.write(self._prompt + " ")
        self._stdout.flush()
        while True:
            ch = msvcrt.getwche()
            if ch in "\r\n":
                with self._lock:
                    buf = u"".join(self._buf)
                    self._buf = []
                buf = normstr(buf)
                return buf
            else:
                with self._lock:
                    self._buf.append(ch)

GFig GIMP 2.6 files parsing/rendering

See modules "gfig" from "HMI pybase" project on Projects page.

Parsing
GFig files are very simple vector format in Gimp 2.x used by GFig plugin for shapes creation. It looks like this:

GFIG Version 0.1
Name: Simly\040face
Version: 0.000000
ObjCount: 1
GridSpacing: 30
GridType: RECT_GRID
DrawGrid: FALSE
Snap2Grid: FALSE
LockOnGrid: FALSE
ShowControl: TRUE
66 150
66 200

Parses only KNOWN SHAPES, all other are ignored. Also parses styles and extra parameters. Suppose fixed structure of file: lines are not joined! Known shapes are:
  • line
  • rectangle
  • circle
  • ellipse
  • poly
  • star
  • spiral
  • bezier
  • arc

Rendering
GFig GIMP 2.6 files renders with Tkinter (on canvas). Supports:
  • lines
  • circles
  • ellipses
  • rectangles
  • polygons
  • arcs
  • old style spirals (via lines)
and it's outline-, fill- colors, line width; also polygon has style ruby (recreate canvas object when changed - interesting effect with smooth=1, ruby=True). Not supported: Bezier, spiral, stars. All unknown objects are ignored.

Transforming of objects is supported: move, scale, rotate. It's possible to transform only one object or any selected objects (set) of gfig-file. First, shape class methods are used, for second, GFigRender methods are used. GFigRender transformation methods has arg shapes - shapes to be transformed. Also, you can select GFigRender shapes by find_shapes() method and process it (transform, configure, for ex.). Finding use classname of shape ("CanvasArc", "CanvasEllipse", etc) or any option (Tk cget, gfig option) in **kw form, or name of shape. Usually shapes don't have names (gfig-extension feature), but user can name each shape in order from left-to-right and top-to-bottom (canvas location) with method name_shapes(). find_shapes() returns list, ifind_shapes() - iterator.

Each shape can have label - text short string. Technically, label is created on first labelconfigure(), each other only changes label options. Label is placed by options: side (N|NW|CENTER...) and standard Tk anchor option; so side specified corner (about shape bbox) or center of bbox

Contribs
Thanks for creators of Gimp GFig:
  • Andy Thomas
  • Spencer Kimball
  • Peter Mattis

четверг, 26 мая 2011 г.

Wireless configuration in Windows Vista

Change in registry branch "HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\ndisuio" value "Start" to 2. Then start services "Wired autoconfig", "WLAN Autoconfig" (first set to "Auto" starting mode - for autoconnecting - and now start them manually). If they doesn't start then see "Dependencies" tab and turn-on - to auto start mode all sub-services) and try again.
After this you can try to find a nets and to connect to one of them (see names of found networks). To connect you need to enter SSID and key phrase - see them in your Wireless Router config :)

See also:

вторник, 24 мая 2011 г.

Linux box on Windows

It's frequent task for programmer: developing of application for alien OS. If you use Windows as host system and need Linux box, there are several free solutions: VmWare Player, VirtualBox, QEmu for your pleasure :)

But after done with virtual machine, you should to create some "bridge" to data exchange with guest OS (ssh/scp, samba/nfs, ftp, shared folder...). When you use VirtualBox, for example, you should install additional software to guest OS. But there is simpler solution: qemu with virtual FAT folders.

Download from QEmu on Windows ZIP file with qemu (yes, it's portable, and is only about 4Mb in ZIP!). Uncompress.

Suppose, we need only developer's box: no X, only GNU toolchain.

Let's try Archlinux, see: www.archlinux.org (download "CORE" ISO image).
Then create image - space for VM:
qemu-img create linux.img 1G
1G is enough room. This will be done quickly.
Then let's boot from ArchLinux CD image:
qemu -m 256 -hda c:\vm\linux.img -cdrom c:\vm\archlinux-2010.05-core-i686.iso -boot d -L c:\qemu\bios
We suppose user uncompress qemu to c:\qemu and puts images to c:\vm. After boot you should install ArchLinux - it's easy, but long :)
OK, after installation you can run Linux box with sharing some folder from host OS. Create first some folder, for example, c:\vm\shared. Then run qemu:
qemu -m 256 -hda c:\vm\linux.img -L c:\qemu\bios -hdb fat:rw:/vm/shared
QEmu creates virtual drive (hdb) from host folder "c:\vm\shared", and you can mount it in Linux box:
su
mkdir /shared
mount -f vfat /dev/sdb1 /shared
"Read-Write" option in qemu is dangerous, so more preferable is to run qemu without "write" mode. For example, in current (0.13.0) version of Windows qemu build there is error: if you add some text to file in /shared, this file will be deleted (on host, yeh:). Use it for read-only access, and never modify shared folder on host when it is mounted in the guest OS.

Have a fun!