Embedded Flight Controller Tutorial

This is an excellent series. I'll post the links here because I keep losing them:

• Project 1: Custom Flight Controller Part 0: Overview
• Custom Flight Controller Part 1: A Simple Revision on Quadcopter Flying Principle
• Custom Flight Controller Part 2.1: Getting the Orientation --- Get Data from MPU9250 Sensor Module 
• Custom Flight Controller Part 2.2: Getting the Orientation --- The Quaternion Kalman Filter
• Custom Flight Controller Part 2.3: Getting the Orientation --- The Matlab Simulation 
• Custom Flight Controller Part 2.4: Getting the Orientation --- Migrate to STM32 
• Custom Flight Controller Part 2.5: Getting the Orientation --- QKF on STM32 
• Custom Flight Controller Part 2.6: Getting the Orientation --- Using Interrupt to Get Data from MPU9250 
• Custom Flight Controller Part 2.7: Getting the Orientation --- Mag Calibration and Overall Algorithm Structure. 
• Custom Flight Controller Part 2.8: Getting the Orientation --- Processing Orientation Visualization

And here's a link to the code on GitHub.

Thank you Lyfkyle, great work!

Using Java 8's Function to refactor Decorator and Adapter patterns

Two days into using Java 8 (I know, long overdue, but you know... App Engine), I'm having fun refactoring some legacy (Swing, ouch) codebase.

One of the most powerful new tools is "Function", and I'm discovering how you can refactor entire classes, anymous or not, with one line of code, see for example this question:

Is it possible to pass parameters, or access external parameters to an anonymous class? For example:

int myVariable = 1;

myButton.addActionListener(new ActionListener() {
    public void actionPerformed(ActionEvent e) {
        // How would one access myVariable here?
    }
});

Here's my proposed solution, using Function:

Function<Integer,ActionListener> printInt = 
    intvar -> ae -> System.out.println(intvar);

int myVariable = 1;
myButton.addActionListener( printInt.apply(myVariable) );

I think the power of Function becomes obvious when we apply it to refactoring patters such as Decorator, Adapter or other type of proxies.

Here's some code I've just refactored, showing how you can refactor an entire class with a one liner Function:

PS Don't miss this talk by Trisha Gee

Transposing vectors of complex numbers in Matlab

Careful when applying the transpose (apostophe) operator to a vector of complex numbers. See here:

You have to use .' (dot transpose)
If you use the tranpose operator on a complex-valued vector, matrix in MATLAB, it returns the conjugate transpose by default.

Correlation vs Convolution

This is a basic relationship in DSP.

When we talk about similarity between signals, we normally talk about "correlation".
When we talk about filtering, in the time domain that's a "convolution".

Correlation and convolution are basically the same thing.
The only difference is that correlation is a convolution by the inverted (or flipped) version of the same signal in the time domain:

y(n) = h(n)*x(n) --> convolution
y(n)=h(n)*x(-n) -> correlation

In Matlab/Octave that would be:

y = conv( h, x) for convolution

or

y = conv( h, flip(x)) for correlation.

In the frequency domain, the operation of flipping in the time domain translates into the "conjugate" version of the transform, that is with an inverted sign for its imaginary part.

Fix for FR_NO_FILESYSTEM error with the embedded FatFS file system

FatFs is a popular file system for embedded applications.
Occasionally though I was having trouble accessing some SD cards, a problem that's been reported here.and here.
I've looked at a working SD card and a not working one.
They were different models and size but the problem came down to the second card not having its logical partition defined as primary. Using Minitool Partition Wizard you can fix this by doing:

1. Right Click on First Partition
2. Select Set Partition as Primary
2. Click on Apply Changes

The partition goes from Logical to Primary.

FAT32 also works, I've tried up to 8GB.

How to reduce memory usage when marshalling large KML files with JAK (the Java API for KML)

This is something I discovered a while ago and never got around to publish it on this blog.

for a while I've been experiencing memory issues when marshalling large files.

I've monitored this usage with some crude profiling:

25,000 locations: 72MB

50,000 locations: 140MB

So I've looked for ways to reduce this problem. One approach is to marshal the file in chunks instead of all at once. Here are some useful links:

http://stackoverflow.com/questions/1775202/how-to-stream-large-files-using-jaxb-marshaller

http://rezarahim.blogspot.co.nz/2010/05/chunking-out-big-xml-with-stax-and-jaxb.html

Example:

JAXBContext context = JAXBContext.newInstance(type);

Marshaller m = context.createMarshaller();

m.setProperty(Marshaller.JAXB_FRAGMENT, Boolean.TRUE);

    java.io.StringWriter sw = new java.io.StringWriter();

    XMLStreamWriter xmlOut = XMLOutputFactory.newFactory().createXMLStreamWriter(sw);

  

    xmlOut.writeStartDocument("UTF-8", "1.0");

    xmlOut.writeStartElement("kml");

    xmlOut.writeDefaultNamespace("http://www.opengis.net/kml/2.2");

    xmlOut.writeNamespace("atom", "http://www.w3.org/2005/Atom");

    xmlOut.writeNamespace("kml", "http://www.opengis.net/kml/2.2");

    xmlOut.writeNamespace("gx", "http://www.google.com/kml/ext/2.2");

    xmlOut.writeNamespace("xal", "urn:oasis:names:tc:ciq:xsdschema:xAL:2.0");

        

    xmlOut.writeStartElement("Document");

  // iterate through your placemarks here

    Placemark placemark = new Placemark()

   ...

    m.marshal(placemark, xmlOut);

 

     xmlOut.writeEndElement(); // Document

     xmlOut.writeEndElement(); // kml

     xmlOut.close();

    

This is an intermediate solution that sacrifices elegance but this way I've been able to reduce memory usage at least by 60%:

25,000 locations: 20MB

50,000 locations: 40MB

I believe a similar approach can be used when parsing large documents.

I hope someone finds this useful

My journey from Python to Java, to Scala... and back to Java

Here's an update on my work for improving the Construct library in java and make it more type safe and easy to use 
https://github.com/ZiglioNZ/construct

Previous versions would work like this.
1. define a Struct at runtime, and composite it by passing in static constructors of other Constructs:

Struct s = Struct( "struct",
     UBInt8("a"),
     UBInt16("b"),
     Struct("foo",
           UBInt8("c"),
           UBInt8("d")))

2. then we can use this struct to parse a byte array. That returns a Map type called Container:
Container c = struct.parse(ByteArray(1, 0, 2, 3, 4));

3. we can then extract the parsed value from the map/container:
assertEquals( 1,  c.get("a"));
assertEquals( 2,  c.get("b")); 
assertEquals( 3, ((Container)c.get("foo")).get("c"));
assertEquals( 4, ((Container)c.get("foo")).get("d"));

You can see there are a few problems that make this API not so nice to use: 
first, the problem with maps and key strings, one has to remember the names and refactoring for those names is painful
second, all that casting: due to java's lack of type inference, it's difficult to deal with HMaps, of the type that parsers return, therefore casting has to be added.

As a way to mitigate these problems, I've looked at Scala.
Scala offers nice things like elegant default constructor and case classes. Those two things combined could help creating a CLASS, instead of a runtime collection of fields. 
Having a class would help the IDE with code completion: no longer I would have to remember field names, they would be class fields, therefore IDE code completion and refactoring.

I've thought long and hard and tried different things, and stumbled against some limitations of scala, case classes and inheritance.
At the end I went back to using Java reflection, with the idea that later I will look into Scala macros to improve it.

How have I done it?
Well, first instead of passing objects to a Struct at runtime, I statically define a Struct with a number of fields:

    class Foo extends Struct {
      public Foo(String name ){super(name);}
      public UBInt8 c;
      public UBInt8 d;
    }

    class S extends Struct {
      public UBInt8 a;
      public UBInt16 b;
      public Foo foo;
    }

Then I've added code to the Stuct constructor that at runtime uses reflection to inspect the Struct fields and create an instance of it, by passing the name of the field, that is also inspected via reflection.

The last trick is a way for each field to hold an Object, that is the result of a call to parse(). The Struct itself updates this value for each field, after parsing.
Now I have a get() method that returns that value for each field, see:

    S s = new S();
    s.parse(ByteArray(1, 0, 2, 3, 4));

    assertEquals(1, s.a.get());
    assertEquals(2, s.b.get());
    assertEquals(3, s.foo.c.get());
    assertEquals(4, s.foo.d.get());

I think it's much nicer to use. It's not type safe yet, since I need to update all my field definition in order to return the correct type, but I'm definitely getting there.

I think this is a win for java, that is still kicking.
+Pascal Voitot Dev can definitely explain all the subtleties why scala reflection would be better than java's, and why macros would be even better. 
From the practical point of view I think this is a good compromise,


https://github.com/ZiglioNZ/construct/blob/83c01e14e30f7c04875f303cd423eecd14d97834/src/main/java/com/sirtrack/construct/Core.java

https://github.com/ZiglioNZ/construct/blob/83c01e14e30f7c04875f303cd423eecd14d97834/src/test/java/com/sirtrack/construct/ConstructTest.java