- – - – - – - – - M I S H A E G – C O I N E N – - – - – - -

In debugging a particular problem when using someone else’s framework, one often needs to head to documentation for help. At this point one can go to the documentation, or google his exact problem. I tend to go for the latter as a first attempt. Then there are times where I do both for hours and still cannot find the solution though docs or google hits. These are really frustrating, so when I finally fix them I’ve decided to post the solutions here in case they can help someone else. They may need clarification. Just ask and I’ll quickly respond if I can remember.

This is a basic outline that I will expand on. I just finished debugging this stuff and want to write the main points before I forget my references and main issues.

Summary: two basic gotchas of full duplex (record from mic and output to speaker) audio IO on the iphone that I ran into are:

1. The AudioUnits IO callback requires you to handle input buffers and output buffers in a very different fashion for each, whereas in the easy to setup core audio callback I used on OS X, they were handled in a similar fashion (both buffers as parameters to the callback). As they are buffers, mishandling them as I did caused my program to crash. Solution: use the parameter buffer for output, and AudioUnitRender() to obtain the input.
2. Sometimes my callbacks were working, verified by my printf debug statements, but no audio output could be heard – the speakers were silent despite a perfectly good-looking setup and debugging session. Cause: DC output or some other condition caused the iphone speakers to go into a muted state even when volume was on. Solution: plug and unplug a headphone.
3. When recording from the mic the audio will default to the headset speaker by the earpiece instead of the large and much louder main speaker by the dock. Cause: this is an apple default. Solution: Use AudioSession API to change the default.

Read on for the details.

I’ve implemented Audio IO using the native API on a few systems including old mac (pre OS X), new mac (Core Audio), and Windows. But since then I’ve been getting lazy and just using portaudio for everything, since it does what I want and gets a reasonable latency in most cases.

I just finished implementing my audio IO for an iPhone app I’ve been working on, and I somehow managed not to kill myself. The documentation is not very good, but I don’t think that’s the main issue I was running into. What lost me about 10 hours of time was a few gotchas that I hope writing here will save others some time.

The basics.
To start off, I wanted a reasonable low latency audio IO with callbacks like I am used to. On the iPhone it seems you need to use AudioUnits for this.
At first I tried to implement an AudioGraph with a Mixer and a RemoteIO audio unit in that, but after a while I realized I could just use a single RemoteIO and no graph. See this site by Michael Tyson for more info. This one will show you how to set up an audio IO mic-in headset-out RemoteIO unit that is probably suitable for most applications that need recording. However I needed to modify it a bit for my purposes.

There are a number of other useful sites for introduction to iPhone audio IO including:

• Tim Bolstad made a pretty legit and working (I used this to start) audio IO example that uses Audio Graphs. As I said, this was over-complicating my situation so I stopped using this, but if you need more than a few AudioUnits, this is the way to go.
• AurioTouch is a pretty thorough example by apple, but due to its complexity I recommend the above two guides first.
• Nobihisa Nagano is a Japanese guy (that is also a supercollider user) who can provide a different approach to AudioUnits if you happen to read Japanese. Perhaps google translate can also help you here.

It’s actually getting quite late, so I’ll stop here for now and hopefully continue later.

Today I had to publish an iOS app written in actionscript. There are a number of almost good guides like this one, which would be good if they just had more pictures.At first I used Flash Builder 4.5.1, which didn’t do the trick. I then used CS 5, and had everything building and running fine via the emulator. But when it came time to publish after generating the provisioning profile it just wouldn’t work. CS 5 kept saying “could not access the digital certificate. could not load keystore file (password may be incorrect)”. I recreated the .p12 file over and over and checked that I was using the distribution certificate instead of the dev one.
Eventually I thought maybe that my password was too complex, having characters like ‘!’ and ‘$’, but these are normal password chars, so I just stupidly assumed that this was not the issue. An hour later after still having no solution I changed the password to simple alphanumeric chars, and it worked. I didn’t see any google links from my searches that lead me to this fix, but I’m posting it here. This is a pretty lame bug, adobe.

There are really no good tutorials for this that I could find, so here’s a tutorial on how to get justin.tv up on ubuntu. This should work with other linux distros. Before you start calculating the millions you’ll be making from ad revenue, lets first get you working with a stream. Then you can broadcast your sick rts skills or live coding sessions, as you see fit and rake in the dough.

http://apiwiki.justin.tv/mediawiki/index.php/Linux_Broadcasting_API

is the main site for how to do this, but it isn’t very detailed and doesn’t go over desktop streaming.

We’re going to use ffmpeg to stream the desktop and capture our audio as well.

1. First you need a justin.tv account, so go create one.

2.Then go to this page (while logged in) and click the ‘show’ link to view your stream key. This is sort of like a password, so don’t give it out.

3. Install ffmpeg if you don’t have it (you probably do,) but if you don’t for debian/ubuntu it’s the following:

sudo apt-get install ffmpeg libavcodec-extra-52

4.Start the stream.

INRES="1920x1080" # input resolution
OUTRES="1024x576"
FPS="20" # target FPS
QUAL="fast"  # one of the many FFMPEG preset
STREAM_KEY=live_231xxxxxxxxx #get your stream key as described above.

Then run ffmpeg with

ffmpeg -f x11grab -s "$INRES" -r "$FPS" -i :0.0  -f alsa -ac 2 -i hw:0,0 -vol 4096 -vcodec libx264 -vpre "$QUAL" -s "$OUTRES"  -acodec libmp3lame -ab 128k -threads 0   -f flv "rtmp://live.justin.tv/app/$STREAM_KEY flashver=FMLE/3.0\20(compatible;\20FMSc/1.0)"

I modified this command from a forum post that used the one below. My changes were to not use pulseaudio and to boost the volume to twice as much (256 is default, so adjust accordingly)
some people had success with pulseaudio, but my ubuntu/wine config must not be setup to use it.
If audio fails, try this command instead.

ffmpeg -f x11grab -s "$INRES" -r "$FPS" -i :0.0  -f alsa -ac 2 -i hw:0,0 -vcodec libx264 -vpre "$QUAL" -s "$OUTRES"  -acodec libmp3lame -ab 96k -vol 4096 -ar 22050 -threads 0   -f flv "rtmp://live.justin.tv/app/$STREAM_KEY flashver=FMLE/3.0\20(compatible;\20FMSc/1.0)"

The above command uses your audio output (what comes out of your speakers) as the audio to be streamed. To record from a mic, you can use pulse (I think it is installed by default):

ffmpeg -f x11grab -s "$INRES" -r "$FPS" -i :0.0  -f alsa -ac 2 -i pulse -vcodec libx264 -vpre "$QUAL" -s "$OUTRES"  -acodec libmp3lame -ab 96k -vol 4096 -ar 22050 -threads 0   -f flv "rtmp://live.justin.tv/app/$STREAM_KEY flashver=FMLE/3.0\20(compatible;\20FMSc/1.0)"

This command should start your stream on justin.tv.
I am not sure why my audio is 1/16th what it should be and I need to scale the volume that much.

Next tutorial will be about how to integrate a webcam into a desktop streaming setup.

Update: I installed the pulse audio device/volume selector, and then ran the second command which uses pulse. While the stream is open I need to change from duplex stereo to output analog only, then change back. It’s weird that this fixes the filtering/low volume issue, but it does get around it. I only needed to make the swap once and it all works great.

sudo apt-get install padevchooser

I have recently switched to youtube because justin.tv/twitch tv doesn’t support saving or transfering videos to youtube if the content is not games. So while I may still use twitch for starcraft 2 and live streaming, I now use the following very similar command to record to mp4, then upload. I find I get much better framerates like this.
This command records the file to my videos folder with a timestamp attached. It wouldn’t take much more work to automate this upload process.

ffmpeg -f x11grab -s "$INRES" -r "$FPS" -i :0.0  -f alsa -ac 2 -i pulse -vcodec libx264 -vpre "$QUAL" -s "$OUTRES"  -acodec libmp3lame -ab 96k -vol 4096 -ar 22050 -threads 0 "$HOME/Videos/JLPTN1/cast720-$(date +%Y_%m_%d_%k_%M).mp4"

For some reason I needed $HOME instead of ~ to get the filepath to work.

I have an old powerbook g4 (2004,) which was my main computer until last year. Apple products are expensive but I think it was pretty worth it’s money. Now I have a Core Duo (2006) macbook and a sweet homebrew core 2 quad pc whose main purpose in life is to fix audacity bugs on windows and make my actions per minute in starcraft higher. It is connected to my sound system so I want to be able to hear my library music from it. Little did I know, this small desire would bring to an epic smb quest.

WIFI is broken on both my new(er) macbook and pc, so I use the old powerbook as a network hub, sharing internet from wifi via ethernet. I want to use it to share files between mac and pc, so I tried using the “Windows File Sharing” option in the Sharing panel, which just boots up an smb server.

However it doesn’t work with PCs and has apparently crappy authentication. There are numerous posts about people trying to set the authentication to ntvlm1 by modifying the smb.conf. I tried this, and many other similar things, to no avail. Then I came across one solution which is to compile and install the latest smb.

It works now after messing with the damn thing for a good 4 or 5 hours. Since there are no obvious other guides on the net for the few folks who want to do this on mac, here are the steps and problems i ran into. You’ll need basic knowledge of the terminal and probably need the xcode developer tools installed to do this.

1. I typed “smb source download” and got version 3.2.6 or so.

Next steps in the terminal
2. Then I cd’d to the directory “source3″ in the smb folder and typed

./configure --prefix=/usr;make;sudo make install

-note: there is also “source4″, but i believe this is an unstable branch.
-2nd note: also don’t use –prefix=/sur/local/sbin like some guy on a forum posts says he does because this will end up creating an (additional) sbin and lib directory within /usr/local/sbin.
-note: this takes about 15minutes to half an hour on a slow pc.

3. Now we need to edit the configuration file for smb.
In the terminal type:

emacs /usr/lib/smb.conf

and change

passdb backend = opendirectorysam

(or whatever it is) to

passdb backend = smbpasswd'

4.Above that line add 2 new lines :

smb passwdfile = /usr/lib/samba/smbpass.passwd
security = user

5.I also changed

guess account = unknown

to

guess account = youruseraccountname

(replace it with your user name of course)

6.Now save with control-x then control-s. quit emacs with control-x control-c

7.Now it’s time to start the client. in the terminal, type

sudo smbd -d

which starts the smb daemon.

8.You now need to add the password (this can only be done when smb is running, apparently.) Type

sudo smbpasswd -a yourmacusername

then enter whatever password you want (This can be different from your mac password.)

9.Connect from your mac/pc. If your ip is 192.168.1.50, then connect to smb://192.168.1.50/yourmacusername. On windows you can do this by mapping a network drive when you right click ‘My Computer’. On mac it’s in the ‘go’ menu->connect to server

-note if things don’t work out for you and you want to test and see error messages, use

sudo smb --debug-level=10 -i

if you do this you’ll need to kill it before starting the daemon (and vice versa). You may also need the kill command if smbd hangs.

I have some additional steps because I wanted to share an external USB drive, which is not possible by using symbolic links or mac aliases in your public folder.
to do this:
1.edit smb.conf again and add the lines

[extusb]
comment = extusb
path = /Volumes/My Passport
browseable = yes
read only = no

Replace “My Passport” with the name of your drive.
To access it just go to smb://192.168.1.50/extusb (if your ip is 192.168.1.50) and use the same password you provided above.

Then everything pretty much works. The latency is pretty crappy so I had to adjust VLC file cache size to play audio/video well. It’s probably not too hard to configure smb more to handle this, but I’m feeling smbd’d out right now. If someone figures it out thought let me know.

Some last notes:
My smb system was a 1.5ghz g4 on 10.4.11.
My smb clients were a macbook core2duo 2.0 ghz on 10.6.4 and 2.4 gHz Core 2 Quad on Vista x64
I read somewhere on a forum that the smb/opendirectory that comes with your mac is a modified version and not compatible with the newest public smb downloads.
I also kinda sorta know the proper way to handle daemons are with launchctl but I don’t know how to set it up, and I never restart this mac, so I haven’t looked into it.

The MAX(a,b) macro bug I posted last was something I had done before. Today and Yesterday I spent a good few hours running over my old mistakes.

What’s a difference between:

unsigned char *dynArray = malloc(500);
unsigned char trueArray[500];

well, Today’s arrrrrrg is that sizeof (dynArray) is just the size of a pointer, not the size of the array as you would get when doing sizeof(trueArray).

assert(sizeof(dynArray)  == sizeof(unsigned char*));      //ok
assert(sizeof(trueArray) == sizeof(unsigned char) * 500); //ok
assert(sizeof(dynArray)  == sizeof(trueArray));           //fails

It’s likely that you knew this, and I had learned this at one point, and thought ‘hmm, how interesting’ while scratching my chin. But I have obviously forgotten it as I was using sizeof(dynArray) to iterate/initialize my array. I have to give credit to the phrases pointer-array-equivalence and somehow my comfort of using malloc(sizeof(*myPointer)) to confusing me into the conclusion that sizeof(dynArray) would really evaluate to 500. I remember writing that line and saying to myself rather convincingly that this is the right way to get the size of an array. It thus royally sucked debugging the actual code since I tend to start reverting the snippets that I am least convinced of first. But debugging the confusion process is actually quite nice. And now I’m pretty sure I’m not going to do that gotcha again. That’s my consolation sherlock.