Uncategorized

Portballintrae

New site

New website featuring my projects in a more organized manner. I’ve copied over the content from the soundofsyntax.wordpress.com blog.

A look back on my work with ScalaCollider

I have just recently completed my first project using ScalaCollider. ScalaCollider is a front-end written in Scala, a hybrid object-oriented/functional language that runs on the Java virtual machine. I would like to share my impressions of using ScalaCollider now that I have come to use it for quite some time. I will mostly compare it to SuperCollider (using SCApp on the mac).

Positive:
– I can use all the infrastructure around the JVM and Scala. It’s easy to manage dependencies and to bootstrap a project using Simple Build Tool (SBT). There are several extremely competent IDE’s available that are used by developers at companies like Twitter. There are performance analysis tools available that are used by professionals. It’s possible to build an executable that can be distributed and that runs with a simple doble-click easily. This was possible in SuperCollider using standalones but it’s a complicated process, which meant I didn’t do it often enough. With scala it’s so easy that I keep binary copies of my project during development at several stages and after each concert (it’s one command in sbt). I also keep a copy of scsynth and the plugins which means that I have an executable that is exactly the same setup I had at that particular concert.
– I personally use the Intellij IDEA IDE, which is amazing. Coming from SuperCollider.app, IDEA just blew me away. It’s easy to edit code, it can reformat code easily, refactor code (change name of variables, classes, etc across multiple files), there’s no more tabbing and hitting space to format text, just place the cursor anywhere on screen and start writing. Because scala is a static language, the autocompletion is mind-boggling at first (in a good way !). Also help is available just by hovering the cursor over class name and hitting a key combination. Help is also available during auto-completion, so it’s easy to check what each method available does (at least after the help system is configured correctly, which might take some time).
– Scala is a new language that is still evolving. It has many new features that are ground-breaking such as implicit conversions, which allows having type classes like in Haskell and extending libraries that we cannot change. It’s a cutting-edge language, full of new ideas and possibilities. SuperCollider in comparison feels old. For instance, the scalaz library is incredibly powerful and advanced. It’s basically pure math. It’s quite a challenge to use it, but what you get in return is a new way of looking at things. Through scala I came in contact with a lot of new concepts (realized in libraries that I can use) such as Option/Maybe, Algebraic Data Types, Futures/Promises, Monads, Applicative Functors, Functors, Foldables, Functional Reactive Programming (Events and EventStreams), etc.
– Even if scala is a language that compiles down to byte-code, it’s still possible to use it as if it was interpreted using a REPL (Read Evaluate Print Loop), which is an interactive prompt where we can evaluates snippets of code. It’s therefore possible to write small scripts without declaring any classes. The environment to do this is not as nice as in SuperCollider, for instance, it’s not currently possible to select a portion of code and evaluate that, although that is just because no one implemented that yet (although sciss has tried already to make a plugin for IDEA to do this). I wouldn’t use ScalaCollider for live coding at this point, since the type system can become cumbersome when trying to code things quickly, although most of the times it’s quite invisible.
– It’s possible to use any Java or Scala library together with ScalaCollider that means that there is already code out there to do almost anything that we can think off, and it’s easy to interface with those libraries because they are all written either in Java or Scala (no need to go down to c or c++). I was astonished when I implemented the functionality of discovering my iphone when using TouchOSC with Bonjour/Zeroconf with just a few lines of code by importing a java library. The same was true of json parsing. It’s possible for instance to do graphical stuff using processing (Java) or SPDE (a Scala version of processing). The possibilities are endless. And most of the times these libraries are actually written by computer scientists, or people that work in companies or institutions, so the quality standards are quite high.

Negative:
– Some things are missing, such as Routines (there are some implementations available, but not final yet) and Patterns and others are still work in progress. Most of what is implemented is already working quite well though, I detected few bugs.
– It takes a bit of time to setup the development environment. Things are quite smoother now then some months ago though, and there is more documentation available to guide us through the process. It took me quite some time to understand how to get the help system going. Now the latest version of the SBT plugin automatically pulls the source jars when generating an IDEA project, which means I get the help there automatically. It took me some time to figure out how to use the continuations plugins in IDEA, which is necessary to use Routines. Hopefully all of this will be ironed out with time.
– It’s not as easy to just evaluate pieces of code, and check what is going on in the code by checking the value of variables. This is in part because the coding style used (which I copied from a sample application) has more classes inside classes, which makes the code more organized, but also more difficult to check things just by evaluating expressions. On the other hand, there is a debugger available, something which is quite handy to check things. It’s magic stepping through the execution process and actually looking at the lines. I confess I still don’t fully understand how to properly use a debugger and many times end up putting a lot of println’s all over the place, but that’s more a limitation that I have.
– The Scala language is more complex then SCLang, there are more things to grasp in order to use ScalaCollider. For instance, scala is statically typed, which means that I got a lot of errors about something being of the wrong type (usually collections) and I didn’t always know how to solve this issues. It gets better with time, but still it might take more time to code something (unless off course we are using an advanced feature of scala that actually makes it less code anyway). I don’t think the average SuperCollider user would have an easy time in scala.
– I found the GUI code a LOT more complex than in SuperCollider. The place where I lost the most time was with scala swing, which is a GUI toolkit. It’s badly documented, and it’s just more complex, although also more powerful. Scala swing has very powerful layout managers, so when I started using it I was delighted that I would never type pixel locations in Scala. But after some time it turned out that it actually takes more time to make the layout system do the right thing then actually typing the hardcoded pixel values by hand (provided I would never change those values, which is often not true).
– ScalaCollider runs in an environment (the Scala development environment) which is not specifically made for music. That means that many things are more complex then they need to be for audio work (things like the GUIs, selecting files from disk, serialization of objects, etc).

All in all, using ScalaCollider has been a quite positive experience. If not for anything else, I learned a lot along the way. I also lost a lot (a LOT !) of time, so ScalaCollider is not for those that are in a hurry to get something done. I had to figure out a couple of things on my own which took extra time. For instance to save settings to disk I had to learn on how to do serialization using the JSON serialization provided by liftweb, and to write some custom code for my classes (in SuperCollider all I needed to do was, .writeArchive or .asCompileString).

ScalaColliderZ

ScalaColliderZ is a library that provides some added features for ScalaCollider based on the scalaz library. Scalaz is a library for pure functional programming in Scala, which takes a lot of inspiration from Haskell. It has several type classes such as Monad, Monoid, Functor, etc.

ScalaColliderZ provides some methods (many more could be implemented) to deal with asynchronous tasks. Things such as sending synthdefs to the server or loading buffers are done in ScalaCollider using responder functions: we pass a function to the method that gets executed whenever the task has completed. The problem with this is that it’s not easy to sequence several tasks. For example it’s convenient to be able to load a buffer and send a synthdef and then play a synth when both tasks have completed. Promises are perfect for this. A promise is a kind of container that will have a value of certain type at some moment in the future. This is similar to Lists that are containers that can have multiple items of the same type or Option which is a container that might or might not have an item. It’s therefore also possible to use map and flatMap to get a new Promise from an existing one and foreach to execute an action when a promise get it’s value. Example:

promise{ /* do something that takes some time*/; 4 }.map(_+1).foreach(println)

Currently there are methods to load Buffers (BufferPromise.load) and to send synthDefs (mySynthDef.loadP) that return a a promise:

And an example of sequencing:

ScalaColliderZ also adds type classes instances for GE for Monoid:

I hope to add more stuff in the future.

How to configure Ubuntu 11.10 for using 2 RME PCI Hammerfall DSP MADI cards together

The objective is to be able to use the 2 RME cards as one single 128 channel sound card. Ubuntu 11.10 does not detect the RME HDSPM out of the box, so we need to compile a new vanilla linux kernel obtained from git. It is also a good idea to make a small detour from the instructions below and configure the kernel for low-latency, but I will leave that for the brave souls (although it’s easy).

git clone https://github.com/tiwai/sound
cd sound
cp /boot/config-`uname -r` .config
yes '' | make oldconfig
sed -rie 's/echo "+"/#echo "+"/' scripts/setlocalversion
make-kpkg clean
CONCURRENCY_LEVEL=`getconf _NPROCESSORS_ONLN` fakeroot make-kpkg --initrd --append-to-version=-custom kernel_image kernel_headers
sudo dpkg -i linux-image-3.3.0-rc1-custom_3.3.0-rc1-custom-10.00.Custom_amd64.deb 
sudo dpkg -i linux-headers-3.3.0-rc1-custom_3.3.0-rc1-custom-10.00.Custom_amd64.deb
sudo update-initramfs -ck 3.3.0-rc1-custom
sudo update-grub

Reboot and you should be running the new kernel which should detect the cards. Doing

cat /proc/asound/cards

should list the cards.

Next we need to make sure we can name the cards individually since hw:0 will possible refer a different card after every reboot. To do this we place the following inside /etc/modprobe.d/hdspm.conf:

options snd_hdspm id=foo,bar

To merge the two sound cards into one for ALSA we create a ~/.asoundrc.

In our case we have the hdspms connected to RME’s ADI-648 units via optical madi cables. One of the ADI’s is the wordclock master for the the other ADI-648 and the pci cards, so we need to set the clock source and the inputs source:

amixer -c foo set 'System Clock Mode' 'AutoSync' && amixer -c bar set 'System Clock Mode' 'AutoSync'
amixer -c foo set 'Input Select' 'optical' && amixer -c bar set 'Input Select' 'optical'

Now we unmute the cards:

for i in `seq 1 64`; do amixer -c foo set 'Chn',$i 64 && amixer -c bar set 'Chn',$i 64; done

To get rid of xruns jack should run with realtime priorities (change johndoe to your user name):

dpkg-reconfigure -p high jackd
usermod -a -G audio johndoe

You can check your privileges with:

ulimit -r -l

You can now start jackd for the 128 channels sound card

jackd -R -P 70 -d alsa -r44100 -dmergedcards -p128 -o128 -i128

and have fun !

Thanks to Adrian Knoth and Florian Faber for all the help with the hdspm driver !

It took me about 2 months to compile this information, so I hope this is useful to someone.

OSX internet sharing under the hood

I was curious this week to know what exactly is OSX’s internet sharing doing. It starts 3 processes as follows:

bootpd -d -P
natpmpd -d -y en0 en1
named -c /etc/com.apple.named.proxy.conf -f

From the man pages:

bootpd:  DHCP/BOOTP/NetBoot server
natpmpd: natpmpd is a daemon implementing NAT-PMP. NAT-PMP is a protocol for handling port forwarding requests from clients behind a NAT. It is normally
invoked by InternetSharing(8).
named: named is a Domain Name System (DNS) server.

The internetSharing process is responsible for starting these processes:

InternetSharing:

InternetSharing is the back-end for the Internet Sharing feature. It is
responsible for configuring the network interfaces, the DHCP server
bootpd(8), the network address translation daemon natd(8), and the Inter-
net domain name server named(8). named(8) is run in caching-only mode
and allows the DHCP server to always offer the same DNS server address to
the DHCP clients, regardless of the value of the actual DNS server
addresses.

more /etc/com.apple.named.proxy.conf
// Declares control channels to be used by the rndc utility.
// It is recommended that 127.0.0.1 be the only address used.
// This also allows non-privileged users on the local host to manage
// your name server. An empty control set means the utility is disabled.
//
controls {
};

options {
directory "/var/named";
/*
* If there is a firewall between you and nameservers you want
* to talk to, you might need to uncomment the query-source
* directive below. Previous versions of BIND always asked
* questions using port 53, but BIND 8.1 uses an unprivileged
* port by default.
*/
// query-source address * port 53;

listen-on { 192.168.2.1; };
forward first;
forwarders { 10.0.2.1; };
};

//
// a caching only nameserver config
//
zone "." IN {
type hint;
file "named.ca";
};

zone "localhost" IN {
type master;
file "localhost.zone";
allow-update { none; };
};

zone "0.0.127.in-addr.arpa" IN {
type master;
file "named.local";
allow-update { none; };
};

acl can_query {
any;
};

edit1:
The reason I made this investigation was that I was curious about what in internet sharing causes disruption of networks. Turns out that the DHCP server is configured to through out  ip’s in any of the mac’s network interfaces. If instead it was doing

bootpd -d -P -i en1

then it would give out ip addresses only to clients on the wireless interface. I’m I missing something or why didn’t apple think of this ?

Functional fibonacci numbers

There are many ways to calculate the fibonacci numbers sequence in scala, but I really like this one:

def fibs(n:Int)= {
    lazy val fibSeq:Stream[Int] = 0 #:: 1 #:: (fibSeq zip fibSeq.tail).map{ case (a,b) => a+b }
    fibSeq.take(n).toList
}

#:: is the lazy cons method, and fibs zip fibs.tail creates a stream of pairs with the latest two numbers in the sequence.

In haskell it can be even more compact:

fib :: Int -> Integer
fib n = fibs !! n
  where
    fibs = 0 : 1 : zipWith (+) fibs (tail fibs)

scala 2.9.1 + sbt 0.11 + Intellij IDEA (10/11) + scala continuations plugin

It took me some time to figure out how to compile a project in Intellij IDEA with the scala continuations plugin turned on. I was getting this error when I ran the project:

bad option: -P:continuations:enable

Here is what you need to do to make it work:

1. In your build.sbt file add:

   autoCompilerPlugins := true
   
   addCompilerPlugin("org.scala-lang.plugins" % "continuations" % "2.9.1")
   
   scalacOptions += "-P:continuations:enable"

2. In sbt run

   reload

   and then

   update

3. Generate the Intellij IDEA project using the sbt command

   gen-idea

4. Open the Intellij IDEA project and go to “Project Structure” -> “Modules” ->”Scala” and add the continuations plugin jar file in the “Compiler Plugins” pane. The jar should be in ~/.ivy2/cache/org.scala-lang.plugins/continuations/jars/continuations-2.9.1.jar .
5. Tick the “Enable Continuations” option in the “Compiler Options” pane.

lift-json serialization pretty printing

The built-in serialization of lift-json prints a string with no newlines. I prefer to have a nicely printed string so that I can edit the file if I want. For this end I created a new function

writePretty

:

object JsonPimps {

  def writePretty[A <: AnyRef](a: A)(implicit formats: Formats): String =
    (writePretty(a, new StringWriter)(formats)).toString

  def writePretty[A <: AnyRef, W <: Writer](a: A, out: W)(implicit formats: Formats): W =
    Printer.pretty(render(Extraction.decompose(a)(formats)), out)
}

Intellij – git, visual interactive rebasing, local history and SuperCollider

Wow, I’ve just discovered that the git integration in Intellij IDEA is quite advanced. It even does visual interactive rebasing, something which I couldn’t find in any of the shiny looking (and non-free) osx git visual apps.

It also automatically saves your files on every change and kees a private list of changes for the last couple of minutes/hours, which is basically a capable undo system for programmers.

Finally, today I’ve discovered that Intellij is actually a capable editor for SuperCollider classes. I use it to edit classes, but still rely on the SuperCollider app to run the actual code. I’ve made a basic xml file for supercollider syntax support.

I get for free the nice font rendering of plain text, all the git + local history functionality, tabbed editing, and folder layout with all my files. Also, it automatically strips whitespace, and I can position the cursor anywhere in the file and start typing