Adding Moneydance Transactions with Jython

April 13, 2015 at 12:14 pm (Software) (, , , , )

A few years ago, I decided to start tracking my finances again. After slogging it through with spreadsheets and double-entry for a few months, I went looking for a software option. From my quick survey, the personal finance software market was a wasteland, especially if you’re not willing to give access to your bank account to a web-app.

In that survey, I ran across Moneydance, a Java-based, double-entry, personal finance tool. It caught my eye because it claimed to be double-entry (behind the scenes), and because it sported a developer API (and with python scripting).

Sure, Java-based was a strike (especially on a Mac, which goes out of its way to be Java-hostile), but given that the scripting access probably meant that I could get my data out of the tool, should I ever need to do so, I went for it. I’ve been very happy with it in the interim.

My bank, however, not so happy. It has this tendency to stop giving me access to my transactions after 90 days (meaning I have to remember to download things). Well, guess what happened. Three times.

So I had 3 gaping holes in my transaction data. The good news was, given the PDF statements and a moderate amount of elbow grease (mainly emacs macros), I could eventually wrangle the entries into a CSV format. The bad news was, I had let my copy of Moneydance get old, and now most of the plugins (notably, the CSV plugins) didn’t seem to work anymore.

So I installed the python extension, and got to work.

The python interface, as it turns out, is a jython console. The downside if this is twofold. One, the console is extremely primitive (especially if you’re used to working in, say, ipython notebook). Two, jython is based on a hideously old python (2.1? Srsly?). This makes writing code a bit of a slog. (edit in emacs. Save. Switch to console. Press “load script”. Read error. Repeat).

Because it’s jython, however, it means the entire Java object framework of moneydance is exposed. Infinite Kind makes the javadoc available (both for the 2012-era app, and the current version), which, along with a single blog post by Ric Werme, forms the entire set of developer documentation for python scripting in Moneydance.

Needless to say, it’s a bit minimal. But, with a genuine need at hand (several hundred missing transactions in CSV format), I decided to have a go. After some trolling of the forums, it seemed that the way¬†to create a transaction was as follows:

  • Get a handle to the root account (RA)
  • Get the transaction set from RA
  • Get a handle to the source (SA) and target (category) accounts (CA)
  • Create a parent transaction on SA (txn)
  • Create a split transaction, directing it to CA
  • Add the split to the parent
  • Call txnSet.addNewTxn(txn)
  • Call any cleanup necessary for the UI and/or listeners.

Three hours of paranoid experimentation followed (this is my finances database, after all), in which I arrived at this piece of code:

import time
from com.moneydance.apps.md.model import ParentTxn, SplitTxn, AbstractTxn

ra = moneydance.getRootAccount()
visa_acct = ra.getAccountByName("Visa")
misc_acct = ra.getAccountByName("Miscellaneous")

desc="test desc"
memo="test memo"
amt = 6502 # i.e. $65.02
# Create a date in epoch format
t = (2015, 4, 10, 12, 0, 0, 0, 0, 0)
secs = time.mktime( t )
txdate = long(secs*1000)

new_txn = ParentTxn(txdate, txdate, txdate, "", visa_acct, desc, memo, -1, ParentTxn.STATUS_UNRECONCILED)
new_txn.setTransferType(AbstractTxn.TRANSFER_TYPE_BANK)
txnSplit = SplitTxn(new_txn, amt, amt, 1.0, misc_acct, desc, -1, AbstractTxn.STATUS_UNRECONCILED  )
new_txn.addSplit(txnSplit)
print new_txn
ra.getTransactionSet().addNewTxn(new_txn)

ra.refreshAccountBalances()

Which, to the best that I can see, worked. (here, the best that I can see = I have been running moneydance for a day since importing my transactions, and nothing has yet blown up. I set a high bar, I know…)

Permalink Leave a Comment

Roomba Cleaning

June 9, 2012 at 8:46 am (Uncategorized) ()

One of the supreme ironies of owning a roomba (or 4) is all the time you have to put in to cleaning them.

I recently had someone bring over a Roomba with a bum bump sensor. Since replacing that part requires tearing the bot completely apart, I though I might as well give the unit a good cleaning first.
<

20120609-094631.jpg

Permalink Leave a Comment

Roomba Repair Day

October 9, 2010 at 3:47 pm (Uncategorized) (, , )

Mom’s Roomba has been acting up. Troubleshooting a Roomba has gotten a lot easier since the service manual was leaked. Power down, Hold dock/clean, and press spot 6 times. Voila! Troubleshooting mode. Press dock-dock to get to the bumper check, and voila. The right bump sensor has failed. Easy fix, right?

Well, no. For no good reason I can think of, iRobot both over- and under-engineered this part of the circuit. Though the bump sensor is really just an on/off switch (one on each side of the bumper), it’s implemented with an IR LED transmitter/receiver. The problem is that the transmitting LED degrades over time, and eventually, the bump sensor fails.

The obvious way to fix this is to replace the offending LED—SparkFun carries the right part, for instance. On spectacular hack to perform in the meantime is to swap out a resistor, so as to boost the receiver sensitivity. A genius hack, given that it was done with no schematics, and eventually made use of a through-hole part to replace the surface-mount resistor.

So why did iRobot make use of a cheap IR LED instead of a bump switch? No idea. It seems like a ridiculous case of overengineering to me.

Oh, and Calgary’s electronics supply stores? Useless to me. I’m surprised Solarbotics didn’t carry the part.

Permalink 2 Comments

Interface Groups

July 1, 2010 at 9:01 pm (Software) (, , , )

I hack on OpenBSD in my spare time. (Yeah, that infrequently.) Every year, around hackathon time, I find myself hunting through the guts of the Operating System sources, learning a new trick, or two. Being able to learn like this is the main reason I use open source. This is an example of a typical learn-by-doing session.

OpenBSD has a handy mechanism for assigning network interfaces (which may have decidedly unremarkable names like fxp0, ne1, sis3, and so on) into something called interface groups.

An interface group is simply a one-to-many mapping—a named bucket for one or more interfaces. This is handy for when you, say, assign fxp2 to be your synchronization interface, via:

    # ifconfig fxp2 group sync

In theory, referring to “sync” in a configuration file somewhere would be sufficient to use the correct interface.

The problem is, not every piece of software knows about interface groups. Adding this support means understanding how to retrieve this mapping from userland.

Let’s have a look at the code. There are two places I can think of that do this mapping: ifconfig, and pfctl. Locating their respective sources gives me a hint:

    $ which ifconfig
    /sbin/ifconfig
    $ which pfctl
    /sbin/pfctl

A quick hunt reveals the source files I’m interested in:
/usr/src/sbin/ifconfig/ifconfig.c and /usr/src/sbin/pfctl/pfctl_parser.c

Not surprisingly, the source are similar in both cases. It turns out we can use the We use the SIOCGIFGMEMB ioctl to do this. To see how, we can have a quick peek at the kernel code that handles it.

Looking in sys/net/if.c:

        case SIOCGIFGMEMB:
                return (if_getgroupmembers(data));

So the relevant kernel code is in if_getgroupmembers() which can be found in the same file. It starts like this:

    /*
     * Stores all members of a group in memory pointed to by data
     */
    int
    if_getgroupmembers(caddr_t data)
    {
        struct ifgroupreq       *ifgr = (struct ifgroupreq *)data;
        struct ifg_group        *ifg;
        struct ifg_member       *ifgm;
        struct ifg_req           ifgrq, *ifgp;
        int                      len, error;

        TAILQ_FOREACH(ifg, &amp;ifg_head, ifg_next)
                if (!strcmp(ifg-&gt;ifg_group, ifgr-&gt;ifgr_name))
                        break;
        if (ifg == NULL)
                return (ENOENT);

If the group name (supplied to the ioctl) is not found, ENOENT is returned (via errno).

        if (ifgr-&gt;ifgr_len == 0) {
                TAILQ_FOREACH(ifgm, &amp;ifg-&gt;ifg_members, ifgm_next)
                        ifgr-&gt;ifgr_len += sizeof(ifgrq);
                return (0);                                                   

        }

If (ifgr_len == 0) in the passed-in structure, we simply return the amount of storage necessary to store all the interface group members.

        len = ifgr-&gt;ifgr_len;
        ifgp = ifgr-&gt;ifgr_groups;
        TAILQ_FOREACH(ifgm, &amp;ifg-&gt;ifg_members, ifgm_next) {
                if (len ifgm_ifp-&gt;if_xname,
                    sizeof(ifgrq.ifgrq_member));
               if ((error = copyout((caddr_t)&amp;ifgrq, (caddr_t)ifgp,
                    sizeof(struct ifg_req))))
                        return (error);
                len -= sizeof(ifgrq);
                ifgp++;
        }

        return (0);
}

Otherwise, we store as much of the interface data as we can. Note that to parse the data coming out, we have to use a struct ifgroupreq. We can find this definition in one of the system header files: net/if.h:

    struct ifgroupreq {
        char    ifgr_name[IFNAMSIZ];
        u_int   ifgr_len;
        union {
                char                     ifgru_group[IFNAMSIZ];
                struct  ifg_req         *ifgru_groups;
                struct  ifg_attrib       ifgru_attrib;
        } ifgr_ifgru;
    #define ifgr_group      ifgr_ifgru.ifgru_group
    #define ifgr_groups     ifgr_ifgru.ifgru_groups
    #define ifgr_attrib     ifgr_ifgru.ifgru_attrib
    };

To understand how to use it, here’s a stripped down version of the function used in pfctl_parser.c:

void grouplookup(const char *ifa_name)
{
        struct ifg_req          *ifg;
        struct ifgroupreq        ifgr;
        int                      s, len;
        char                     ifgbuf[IFNAMSIZ+1];

        if ((s = socket(AF_INET, SOCK_DGRAM, 0)) == -1)
                err(1, "socket");
        bzero(&amp;ifgr, sizeof(ifgr));
        strlcpy(ifgr.ifgr_name, ifa_name, sizeof(ifgr.ifgr_name));
        if (ioctl(s, SIOCGIFGMEMB, (caddr_t)&amp;ifgr) == -1) {
                close(s);
                return;
        }
        
        len = ifgr.ifgr_len;

        if ((ifgr.ifgr_groups = calloc(1, len)) == NULL)
                err(1, "calloc");
        if (ioctl(s, SIOCGIFGMEMB, (caddr_t)&amp;ifgr) == -1)
                err(1, "SIOCGIFGMEMB");
        
        for (ifg = ifgr.ifgr_groups; ifg &amp;&amp; len &gt;= sizeof(struct ifg_req);
            ifg++) {
                len -= sizeof(struct ifg_req);
                bzero(&amp;ifgbuf, sizeof(ifgbuf));
                strlcpy((char *)ifgbuf, ifg-&gt;ifgrq_member, sizeof(ifgbuf));
                printf("%s ", ifgbuf);
                /* should do ifa_lookup here */
        }
        free(ifgr.ifgr_groups);
        close(s);
        printf("\n");  
}

To use it, we can slap it in an executable with a main like this (complete source here):

int
main(int argc, char **argv)
{
        if (argc &lt; 2)
                exit(0);
        grouplookup(argv[1]);
        exit(0);   
}

And voila: the code works as expected:

    $ ifconfig ath0 group whatev
    $ ifconfig em0 group whatev
    $ ./a.out whatev
    ath0 em0

Ain’t source code wonderful?

Permalink Leave a Comment

Square Roots

June 19, 2010 at 1:00 pm (Math) (, , , , )

Here’s a computational question: how do I tell if an integer is a perfect square?

Possibly the most straightforward way is to compute an approximation (e.g. via Newton’s meod) to the square root, truncate, then square and compare.

This seems somehow unsatisfying to me. I want an exact answer, and I would like to use only integer operations to get there.

Let’s forget about computers for a minute. It turns out people have been computing integer square roots for a very long time—via an abacus. There are several abacus-based algorithms for computing a square root, but they essentially boil down to this observation:

(a+b)^2  = a^2 +2ab + b^2.

That’s an unassuming little formula, but consider what happens when we start thinking about numbers positionally. In real life, we typically write an integer as a string of base-k digits. We will denote this string-notation using curly brackets; i.e.
\{1234\}_{k} = 1 \times 10^3 + 2 \times 10^2 + 3 \times 10^1 + 4 \times 10^0

To simplify things, when we talk about base 10, we will drop the subscript $k$.

Now think about squaring a two-digit base-10 integer:

\{ab\}^2 = (10a + b)^2 = 100a^2 + 20ab + b^2

A rough approximation, then, would be to guess at the first digit, a, and then choose an appropriate b from the above formula. By iterating this process over successively accurate choices for a, we eventually reach a solution.

Another way to think of this iteration is to consider:

(A+B+C)^2 = (A+B)^2 + 2(A+B)C + C^2

Setting A=100a, B=10b, C=c, we quickly see that

\{abc\}^2 =100 \{ab\}^2+ 20\{ab\}c + c^2

And this pattern works for any length digit string.

This leads to the abacus algorithm, given in [1], which we will look at in a later article.

[1]:http://webhome.idirect.com/~totton/soroban/KojimaSq/

Permalink Leave a Comment

Little-oh

April 24, 2010 at 11:26 pm (Math) (, , )

Why is it impossible to find a nontrivial example of someone using little-oh notation?

Recall, little oh is defined as follows. We write f(x) = o(g(x)) if

\lim _{x \to \infty} \frac{f(x)}{g(x)} = 0.

Let’s say I want to get rid of silly little terms like \log \log x. Notice that:

\log\log x = e^{\log\log\log x}  = e^{(\log\log x) \frac{\log \log \log x}{\log \log x}} = (\log x)^{\frac{\log \log \log x}{\log \log x}}.

Of course, \lim_{x \to \infty} \frac{\log \log \log x}{\log \log x} = 0, so this exponent is o(1).

Using this trick, we can rewrite expressions in this way:

\log x \log \log x = (\log x)^{1+o(1)},

and hence all the ugly little log terms have been swept under the rug.

Permalink Leave a Comment

Follow

Get every new post delivered to your Inbox.