headermask image

header image

CamStudio, Adobe Premiere, VirtualDub, Lossless CSCD and LZOCodec

As far as Windows-based decent free screen-recording software goes, CamStudio is it. When installing, make sure you also download and install the CamStudio Lossless Codec (CSCD)! There are a couple of problems that become apparent when actually using captured video in other video processing apps (e.g. Media Player Classic (MPC), Adobe Premiere and VirtualDub). The following list assumes video is compressed with CSCD, unless otherwise noted.

  1. Under Vista, the ‘Record audio from speakers’ option does not work.
  2. When opening a captured AVI with MPC, no video is shown (but audio is heard if it was recorded).
  3. When importing a captured AVI into Adobe Premiere, all frames of video are blank and only the cursor may, or may not, be seen. Also the following message is found in the event window: “File importer detected an inconsistency in the file structure of <captured.avi>. Reading and writing this file’s metadata (XMP) has been disabled.”
  4. If using the LZOCodec instead of CSCD, VirtualDub will crash if you play the captured AVI or attempt to transcode it (funnily enough, stepping through frames will not trigger the crash).
  5. If audio is captured, frames of audio are heard to be missing when playing back the AVI in any program.
  6. If/once the correct video is seen in Premiere, playback shows jittery, blending of frames and incorrect colours appearing. The base video data seems to be there, but Premiere is not interpreting it correctly and so it appears corrupt.

I recommend the following modifications to resolve the above problems, respectively:

  1. Stick with ‘Record audio from microphone’, but change the ‘Audio Capture Device’ (in ‘Audio Options for Microphone’) to whatever line represents your soundcard’s output mix (e.g. ‘Stereo Mix’). Then adjust the output levels of you soundcard and applications accordingly using the Windows Mixer.
  2. MPC appears to be quite finicky regarding an AVI’s RIFF tree. For some reason, CamStudio inserts additional data after the AVI Legacy Index at the end of the file (I’m not using the term ‘junk’ because JUNK tags are actually valid RIFF tags!). You can see the garbage for yourself by using VirtualDub’s in-built hex editor and opening the RIFF chunk tree – there’ll be an ‘invalid chunk’ after the top-level AVI chunk tree. Although VirtualDub, Windows Media Player and Adobe Premiere can read the files, MPC decides to hide the video. To create a garbage-less AVI, open the AVI in VirtualDub and perform a straight audio and video stream-copy to a new AVI. The new version will be identical to the old, minus the garbage at the end. MPC will successfully playback the video in the new version. This also allows Premiere to read the AVI and not report the ‘inconsistent file structure’ message mentioned in point 3 above.
  3. As was discovered here, Adobe Premiere misinterprets the alpha channel in the captured AVI. To rectify this, right-click on the clip in the bin, select ‘Interpret Footage…’ and enable the checkbox that instructs Premiere to ignore the alpha channel. Upon hitting OK, the proper frame will be shown.
  4. As suggested here, the LZOCodec is an alternative to CSCD. As I discovered, it does have a lot of nice options, including a console debug output and new archival compression option. However the fact that it causes VirtualDub to crash (curiously not MPC, the Windows thumbnail extractor, nor Adobe Premiere) is a serious downer. I have found no workaround – the obviously solution would be to get the source and debug. Until the problem is isolated, I would recommend staying with CSCD if you like VirtualDub.
  5. Losing audio data is a major pain. I found that changing the ‘Compressed Format’ to plain old PCM and interleaving the video and audio every 1 frame (instead of 100 or 500 milliseconds) offered the best result. During limited testing, I couldn’t discern any dropouts with the new settings.
  6. Adobe Premiere, as it turns out, has trouble dealing with certain codecs (in this case CSCD, but interesting not LZOCodec) that store delta frames between keyframes. Delta frames are great because they tend not to don’t duplicate redundant data by using information from previous frames, and therefore decrease the size of the captured AVI file. However when your video editor doesn’t like the way your codec is decompressing them, you don’t have much of a choice except to disable delta frames altogether. This is easily done in CamStudio by changing the ‘Set Key Frames Every’ setting in ‘Video Options’ to 1 (i.e. create a keyframe on every frame). You’ll need to uncheck the ‘Auto Adjust’ option at the bottom of this dialog and specify a capture rate. The default is 200 frames per second, however I use 25 FPS (i.e. capture frames every 40 milliseconds). This will increase the overall filesize, but satisfy Premiere, which will playback the video correctly. Premiere also says the captured AVIs appear to have dropped frames (in a clip’s properties), but this can safely ignored. Perhaps it is the existence of delta frames and empty frames in a file that confuses Premiere, or perhaps it is solely due to a bug in CSCD. Either way, not using deltas will work.

Using the Visual Studio 2005 IDE with the Visual Studio 2008 VC compiler & VC++ libraries

Having recently upgraded to Visual Studio 2008, my lackluster experience with developing and debugging native code in the ‘improved’ IDE has left me looking (and I am still) for hotfixes that might patch various painful problems. In particular, the IDE suffers major performance problems when debugging large C++ projects that (believe it or not) have breakpoints set. With a little experimentation I discovered that actually have any breakpoints at all causes a massive performance hit when single-stepping through one’s code, whether or not they are disabled. This is not a good thing for a debugger and it’s GUI, which is a real shame since Visual Studio 2005 was not plagued by any such issues.

This leads me to the following idea: why not keep using the Visual Studio 2005 IDE for development and debugging, but make it use the VC 2008 compiler and newer libraries (i.e. MFC 9 and the VC++ 2008 redistributables)? This will mean that Visual Studio 2005 will build your projects with the newer compiler and link to the latest versions of the runtime/MFC/etc. Since the 2005 debugger will work for either version (i.e. 2005- or 2008-compiled code), one can safely revert back to Visual Studio 2005 for such simple activities as single-stepping (phew!). I must point out that one good think Microsoft did for Visual Studio 2008 was dramatically improve their performance analysis tools – this would be the only reason to use the 2008 IDE: the newer profiling tools/GUI.

The first thing to do is to identify the full path of Visual Studio 2008’s ‘VC’ directory, and where version ‘6.0A’ of the Platform SDK resides. By default they are (respectively):

C:\Program Files\Microsoft Visual Studio 9.0\VC
C:\Program Files\Microsoft SDKs\Windows\v6.0A

Next you need to create a property sheet that contains the necessary information for a VS 2005 C/C++ project to use the 2008 headers and libraries.
Create a new property sheet file manuall (e.g. “VS9.vsprops”) and insert the following, adjusting the paths where necessary:

<?xml version="1.0" encoding="Windows-1252"?>
<VisualStudioPropertySheet
 ProjectType="Visual C++"
 Version="8.00"
 Name="VS9"
 >
 <Tool
 Name="VCCLCompilerTool"
 AdditionalIncludeDirectories="&quot;C:\Program Files\Microsoft Visual Studio 9.0\VC\include&quot;;&quot;C:\Program Files\Microsoft Visual Studio 9.0\VC\atlmfc\include&quot;;&quot;C:\Program Files\Microsoft SDKs\Windows\v6.0A&quot;"
 />
 <Tool
 Name="VCLinkerTool"
 AdditionalLibraryDirectories="&quot;C:\Program Files\Microsoft Visual Studio 9.0\VC\lib&quot;;&quot;C:\Program Files\Microsoft Visual Studio 9.0\VC\atlmfc\lib&quot;;&quot;C:\Program Files\Microsoft SDKs\Windows\v6.0A\Lib&quot;;&quot;C:\Program Files\Microsoft Visual Studio 9.0&quot;"
 />
 <Tool
 Name="VCResourceCompilerTool"
 AdditionalIncludeDirectories="&quot;C:\Program Files\Microsoft Visual Studio 9.0\VC\include&quot;;&quot;C:\Program Files\Microsoft Visual Studio 9.0\VC\atlmfc\include&quot;;&quot;C:\Program Files\Microsoft SDKs\Windows\v6.0A\Include&quot;"
 IgnoreStandardIncludePath="true"
 />
</VisualStudioPropertySheet>

Then in Visual Studio 2005, open the Property Manager (via the View menu), and add this new file. Make sure in the Project Settings, under General, the property “Inherited Project Property Sheets” includes the new file.

The last trick is to add the following paths to the “Executable Files” list in VS 2005 under Tools -> Options -> Projects and Solutions -> VC++ Directories:

C:\Program Files\Microsoft SDKs\Windows\v6.0A\bin
C:\Program Files\Microsoft Visual Studio 9.0\VC\bin
C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE

The first is for RC.exe, the second is the compiler and the third is for the linker to find DLLs for manipulating debug information (e.g. mspdb80.dll).
NOTE: They don’t have to be added at the very top of the list (e.g. the NASM directory might be at the top) but they must come before the VS 2005-versions of the same directories. That is, they should come before the following since they are supposed to override them (that said, you shouldn’t have to remove any entires if the order is correct):

$(VCInstallDir)bin
$(VSInstallDir)Common7\Tools\bin
$(VSInstallDir)Common7\ide

Also, note: once you put these new paths in, ALL projects compiled under VS 2005 will use the VC 2008 compiler and tools. Unfortunately there is no clean way to override the compiler/tool paths using the VSProps method above. I tried creating a user-defined macro in the VSProps named ‘VSInstallDir’ to override the in-built one, but it had no effect. Although I haven’t tested it, there shouldn’t be a problem if you use the new compiler on VS 2005 projects without the inherited VSProps – you’ll just have a newer compiler linking to MFC 8 and the VC 2005 runtime, which shouldn’t (hopefully) cause any dramas!

Sydney Spring Cycle 2009 GPS Track

I ran InstaMapper for the duration of the ride on my iPhone, resulting in the following track:

GPS tracking powered by InstaMapper.com

Direct link to track.

iPhone Exchange Push Email (ActiveSync) only operates when connected via 3G, not WiFi, when mail server’s IP address is not externally-routable

I host my own mail server (Courier with Spamassassin) and check my mail via IMAP on my iPhone. Unfortunately Apple never implemented use of the IDLE command to enable Push with an IMAP account (there is a jailbroken app that will do this!), so the alternative (if one is desperate for Push) is to pipe their email to a service that is supported (Yahoo, and now apparently Gmail), or set up Microsoft Exchange. Me, being me, I decided to try something new, and set about setting up Exchange. The first step, of course, is installing Windows Server (I decided to do a fresh install of 2008 Datacenter Edition), setting up a domain controller/forest/domain, and then Exchange, which I actually installed on another machine that runs Windows Server 2003 R2. The only issue is that Microsoft has not release a supported version of Exchange Server 2007 for 32-bit x86 machines! The only option is their 64-bit build, and as I don’t have any 64-bit server machines (my servers are a tad old) I had to make do with Exchange 2003 with Service Pack 2 (SP2 includes the additional functionality to enable Push, specifically ActiveSync).

After it all appeared to be installed and happy, my iPhone would not beep, no matter how many emails I sent myself. As it turns out, the entire time (of course) it was connected to my WiFi network. As soon as I disabled WiFi on the device, and it connected to my server via the 3G network, I noticed the “IP-AUTD Initialized” message in the server’s event log. Hurrah!

The reason? Have a look at point #2 in: iPhone 2.0 software: Troubleshooting iPhone or iPod touch Exchange ActiveSync “Push” issues (thank you very much to Tonicwater, see below). Perhaps the iPhone could check for a network environment change (i.e. logging on a WiFi network) and flush the DNS cache – at least for mail/corporate-related activities, such as the address of one’s registered Exchange servers. Then they would always be current, and the ‘push’ process could be re-initiated with the same server at its newfound address.

The final step was to forward mail from my Linux mail server to Exchange. This was accomplished by adding the following code to the end of Courier’s maildroprc file:

if ($LOGNAME eq "(my Courier email address)" || $USER eq "balint")
{
  cc "!(my Exchange email address)"
}

I also added an Exchange recipient policy so that it would only keep the last 30 days of messages, which doubles as a backup of sorts.

Here is a useful page to help with diagnosing problems with Push in Exchange: http://msexchangeteam.com/archive/2006/04/03/424028.aspx

Error when instrumenting files for profiling using Visual Studio Performance Tools

A while ago I started using the Performance Tools that came bundled with Visual Studio 2005 Team Suite to analyse my code. There are two analysis methods: sampling and instrumentation. Although instrumentation has a higher overhead, it can reveal in greater detail exactly what your program is doing. In order for instrumentation to proceed, one must link with the /PROFILE switch and then use VSInstr to insert counters into one’s code. To cut down the cruft collected during a profiling session, one can use the VSInstr switches to dictate exactly which functions should be instrumented and/or use the VSPerf API to command the profiling engine to start/stop/suspend/resume/mark data collection during a run.

One fatal flaw of the Performance Explorer in the 2005 IDE was that one couldn’t start a program with data collection in the stopped state. It would always start collecting data from the very beginning of execution. Visual Studio 2008 includes a command to “Launch with Profiling Paused”, which is a great way to only start profiling when you need to (e.g. via StartProfile or VSInstr /STARTONLY).

Alas, there is still a bug in the Performance Explorer (or, rather, in its build system). If one selects, through the Binary properties of a performance session, to ‘relocate instrumented binaries’, and then proceeds to set ‘additional instrumentation options’ in the Advanced section, the IDE will complain when attempting to start a new profiling session and print this error message in the Output window: “Object reference not set to an instance of an object.”. This is highly annoying because it is helpful to put the instrumented binaries elsewhere (not where the original build outputs reside) AND add specific command line options, such as /VERBOSE and /CONTROL (in conjunction with the others).

I don’t know how to work around this bug; I had hoped it was fixed in 2008, but apparently not. The easiest thing is to not relocate the instrumented binaries. Perhaps one can add another build configuration that includes the /PROFILE linker option and uses its own folder as the build output destination where the binaries can be instrumented in-place.

Here are some helpful resources about using the Performance Tools:
http://blogs.msdn.com/angryrichard/archive/2005/01/16/354194.aspx
http://blogs.msdn.com/scarroll/archive/2005/04/13/407981.aspx
http://blogs.msdn.com/ianhu/archive/2005/06/09/427327.aspx

Drupal request_uri returns broken absolute URLs when served from an Apache VirtualHost

After upgrading my Drupal site to the latest point release (5.19), before transitioning to version 6, I discovered that the action parameter of all rendered forms were being prefixed a slash ‘/’. This in itself is reasonable since one would expect the REQUEST_URI server variable to return a relative URL. However, as I host several websites, Drupal runs inside its own VHost with URL rewriting to enable short URLs (without the ‘q=’ querystring). If I access Drupal directly via the web server’s address, which maps to the default VHost (this also happens to be Drupal), REQUEST_URI is relative. If I access it via the address externally visible on the web (http://spench.net/) then REQUEST_URI is absolute (i.e. prefixed with the protocol and domain).

The Druapl ‘request_uri’ function in includes/bootstrap.inc always adds a slash to the beginning of the server variable REQUEST_URI, which will break any absolute URLs.

The following modification (the strpos check) resolves this issue:

/**
* Since $_SERVER['REQUEST_URI'] is only available on Apache, we
* generate an equivalent using other environment variables.
*/
function request_uri() {
  if (isset($_SERVER['REQUEST_URI'])) {
    $uri = $_SERVER['REQUEST_URI'];
  }
  else {
    if (isset($_SERVER['argv'])) {
      $uri = $_SERVER['SCRIPT_NAME'] .'?'. $_SERVER['argv'][0];
    }
    else {
      $uri = $_SERVER['SCRIPT_NAME'] .'?'. $_SERVER['QUERY_STRING'];
    }
  }
  if (strpos($uri, '/') == 0)
  {
    // Prevent multiple slashes to avoid cross site requests via the FAPI.
    $uri = '/'. ltrim($uri, '/');
  }
  return $uri;
}

The Mammalian Diving Reflex

From Wikipedia:

“The mammalian diving reflex optimizes respiration which allows mammals to stay underwater for a long time. It is exhibited strongly in aquatic mammals (seals, otters, dolphins, etc.), but exists in a weaker version in other mammals, including humans. Diving birds, such as penguins, have a similar diving reflex. Every animal’s diving reflex is triggered specifically by cold water contacting the face – water that is warmer than 21 °C (70 °F) does not cause the reflex, and neither does submersion of body parts other than the face. Also, the reflex is always exhibited more dramatically, and thus can grant longer survival, in young individuals.”

Interesting graph of bradycardia in a diving seal.

Something to think about next time you splash some cold water on your face!

Microsoft Visual Studio 2005 crashes when typing into a code window while debugging

When debugging my VC++ projects, I routinely make use of ‘Edit & Continue’ by modifying code in the Visual Studio environment while stepping through the compiled code. Recently I found that typing into a code window would cause Visual Studio to become unresponsive for a long period. Checking Task Manager, I found that WerFault had kicked in and was generating a minidump for DevEnv.exe – Visual Studio was clearly not happy. The unusual thing was that Visual Studio would not immediately terminate after the minidump finished. Instead, it would continue running. However, if I would now close the debugee (my own application), attempt to continue execution with modified source code (which would activate Edit & Continue), or manually stop the debugger, Visual Studio would crash.

After a very thorough search (looking at Visual Studio’s stack trace using symbols from the Microsoft Symbol Store, seeing exceptions thrown from HeapFree and IDebugEncLineMap, un-installing various plugins such as AnkhSVN, and duplicating the tests on another machine), I could not identify the cause. Thinking about any configuration changes I had made in the last months, I tried removing the all additional Include & Library paths I had ever added to the VC++ directory options. These can be found (on Vista) in:

C:\Users\User Name\AppData\Local\Microsoft\Visual Studio\8.0\VCComponents.dat

Voila! No more crashes! As it turned out, I had added so many extra Include/Library directories (each with a long path) that some portion of the debugger would crash when reproducing the steps described above. Perhaps this is a buffer overflow? I had a look through the patch listing on Microsoft Connect, but found nothing matching this problem.

Now, I wanted to have my cake and eat it too: I resolved this issue, while keeping all of my configured Include/Library directories, by creating a hard-link (or NTFS reparse point) to the base of my SDK directory (under which all of the Include/Library directories reside) further up the directory chain, in effect shortening the path lengths.

For example, here are two Include directories:
C:\Documents and Settings\User Name\My Documents\Visual Studio 2005\Projects\_SDK\Some Code\include
C:\Documents and Settings\User Name\My Documents\Visual Studio 2005\Projects\_SDK\Other Code\include

Lots of these will crash Visual Studio.
They both share the parent: C:\Documents and Settings\User Name\My Documents\Visual Studio 2005\Projects\_SDK

So create a reparse point at: C:\Dev\SDK And make it point to the parent above.

So now in Visual Studio, you can change the original Include directories to:
C:\Dev\SDK\Some Code\include
C:\Dev\SDK\Other Code\include

In total, they are of a much shorter length and fall under the mysterious buffer limit, thereby avoiding the dreaded crash scenario.

NTFS soft/hard-links can be made using the command-line tools at the bottom of this handy page.

Changes in ActionScript code not reflected in published content

While working on a Flash project in Adobe Creative Suite CS4, I was editing some ActionScript 2.0 code (AS) stored in a separate code file from the main Flash document (FLA). I decided to revert to an older version of the whole project, thereby going back to FLA and AS files with an older timestamp. Publishing the document proceeded as normal, however when I went to preview it I found behaviour that would indicate that the newer AS code was still being compiled, even though I had gone back to using old code. After using Process Monitor and highlighting the code file in question, I discovered Flash caches compiled classes (to ASO files) in a special directory (under Windows Vista):

C:\Users\User Name\AppData\Local\Adobe\Flash CS4\en\Configuration\Classes\aso\...

I suspect under previous versions of Windows, it would be the same directory structure under:

C:\Documents and Settings\User Name\Application Data or
C:\Documents and Settings\User Name\Local Settings (If you know, please leave a comment!)

I guess that as the old AS code file had a timestamp older than the cached ASO file, the cache was not refreshed and the original code was used.
There are two options: delete the ASO files, or ‘touch’ the necessary AS files with a newer file time.

T_Visionarium featured on SBS World News

Please visit this page to view the video clip.