TheRealTimeWeb.com - Tools

Move An Existing Git Repository Into Bitbucket In 3 Steps

Wed, 16 May 2012 09:03:00 +0100

Thanks to my good friend Simon I've been a happy git user for a few months now. I installed my own git server on Amazon EC2 using Ubuntu, git and gitolite and just fired it up once or twice a day to push and pull. This worked out fine but with more and more easy to use git hosting services springing up I gave Atlassian's bitbucket a try since it offers unlimited private repositories which is a bonus.

It took me a little while to figure out how to move my existing repository into bitbucket, especially since it was already tracking the existing remote repo on my server. What follows are a few easy steps that describe the process - but note that you follow along at your own risk.

1) Create a new bitbucket repo
Yes, I did not import my existing repo since I could not figure out how to do that. I connect via ssh to my git server yet bitbucket asks for a username andpassword when importing repos that require authentication. Maybe there's a way to do it, but I did not waste any more time on it, and the way outlined here also keeps your project history in tact, and it's simple.
Once you created the new, empty repo on bitbucket simply leave that page open, we'll come back to it in step 3.

2) Detach your existing local git repo from its remote
I'm on OSX so I used Terminal to cd into my local project folder: $ cd '/Users/stefan/dev/projects/myproject'

You can tell git to show you the remote aliases for your project:
$ git remote

To see the actual URL for each alias use $ git remote -v

I then detached my existing remote from the local repo:
$ git remote rm origin

3) Add your existing project code to the new remote repo on bitbucket
Back onto the bitbucket website where you still have the page open. You did leave it open, right? If you didn't then find the option I mention below, or just nuke the project and create it again.
There's a big fat 'I have code I want to import' showing under 'Add some code'. Click it.

Simples. Bitbucket shows you the exact code you need to enter into Terminal to add your project to the new repo. It looks something like this:
$ cd /path/to/my/repo $ git remote add origin ssh://git@bitbucket.org/username/myproject.git $ git push -u origin master # to push changes for the first time

You're done. If you think I'm out of my mind or have overlooked something then please feel free to have a moan in the comments. Or thank me if it helps :-)

Easily Monitor Log File Output in OSX Terminal (tail -F)

Tue, 28 Feb 2012 08:34:00 +0100

Here's a quick tip that is a real time saver in some situations. As a developer you often come across programs, tools or servers that produce output in form of a log file. Or maybe - and this includes myself - you use logs for debugging purposes.

In ColdFusion for example I regularly find myself adding cflog tags to my code in order to get a view into what my code is doing. Having to open and close log files constantly can be tedious at best, but luckily there is a really simple way of displaying a live view of the most recent entries using the standard Unix tail -F command.

Whereas tail on its own simply displays the last part of a file, the -F option will not stop when the end of the file is reached but will keep monitoring the file for new lines and displaying them, thereby giving you effectively a live console view into the file.

Taking ColdFusion logs as an example, here's how you'd monitor a particular log file (mine are typically named according to the site or application I am working on):
cd /Applications/ColdFusion9/logs/ tail -F playingornot.log "Information","jrpp-4","02/21/12","17:40:30","PLAYINGORNOT","/index.cfm" "Information","jrpp-5","02/21/12","17:40:33","PLAYINGORNOT","/index.cfm" "Information","jrpp-5","02/21/12","17:40:35","PLAYINGORNOT","/cfc/Auth.cfc" "Information","jrpp-5","02/21/12","17:40:35","PLAYINGORNOT","/cfc/Players.cfc" "Information","jrpp-5","02/21/12","17:40:36","PLAYINGORNOT","/cfc/PlayTimes.cfc" "Information","jrpp-1","02/28/12","09:37:37","PLAYINGORNOT","/index.cfm" "Information","jrpp-1","02/28/12","09:37:40","PLAYINGORNOT","/cfc/Auth.cfc" "Information","jrpp-1","02/28/12","09:37:41","PLAYINGORNOT","/cfc/Players.cfc" "Information","jrpp-1","02/28/12","09:37:42","PLAYINGORNOT","/cfc/PlayTimes.cfc" "Information","jrpp-2","02/28/12","09:40:30","PLAYINGORNOT","/cfc/PlayTimes.cfc"

As my application runs I can see new entries being displayed in real time in my log file - very handy. This particular log is produced by adding to my Application.cfc's OnRequestStart handler.

This technique works for any kind of text based log file on operating systems that have the tail command or similar available.

Installing git and gitolite on Ubuntu 11.10

Mon, 13 Feb 2012 11:19:00 +0100

Don't ask me how I did it (ok, you can ask, in fact I'm going to tell you...), but somehow I now have a remote git server and gitolite for repo and user administration installed and running on Ubuntu 11.10 using an Amazon EC2 Micro instance.

I've spent all morning on this, tested it on 3 different instances (I love it how you can just throw away an EC2 instance and start again with a new one in a matter of minutes) and have kept a log of the steps which got me there. Please note that there may be errors or illogical jumps in this summary, but maybe it will be helpful nonetheless - or it may even work outright, who knows.

BEfore we get started, here are some links that helped me (but note I had to pick some steps from some links, and other steps from others...). In particular I did not clone gitolite to my local machine (as this post suggests but ended up using sudo apt-get install git

Links I used:
git + gitolite + git-daemon + gitweb setup on Ubuntu 11.10 server
How to setup git server using gitolite in Ubuntu 11.10 Oneiric
Install Gitolite To Manage Your Git Repositories

First of all, get yourself an instance of Ubuntu (you may swap this for your preferred Linux distro). If you are in love with EC2 then head over to alestic.com and click the selection at the top of the page (I'm using the us-east region), and then pick your preferred image.

The neat thing is that if you already have an Amazon Web Services account you can launch your instance with virtually one click, very neat. I picked the Ubuntu 11.10 Oneiric EBS boot image, 64 bit.

Once booted, SSH into your instance (consult other blogs for details on this if you need help).
I then ran the following commands first:
sudo apt-get update sudo apt-get upgrade
Then create a tmp folder
mkdir tmp
which is where you will copy your public key into. Now open another local Terminal window and copy your public key - consult the steps for generating new keys from other sources if needed.
# local machine scp ~/.ssh/id_rsa.pub yourEC2server.com:/tmp/stefan.pub
Note that SCP is not always straight forward - I struggle with the syntax at times - and EC2 complicates it a bit. So the format may in fact look more like this: scp -i ~/.ec2/yourEC2pair.pem ubuntu@ec2-999-239-176-93.compute-1.amazonaws.com:/tmp
I then installed git, gitolite and some stuff which I am not sure is strictly needed:
sudo apt-get install git gitolite git-daemon-run git-doc
Then added my git user details and added the git user
git config --global user.name "Stefan Richter" git config --global user.email "my@domain.com" sudo adduser --system --shell /bin/bash --gecos 'git version control' --group --disabled-password --home /home/git git
The system should reply with
Adding system user `git' (UID 105) ... Adding new group `git' (GID 111) ... Adding new user `git' (UID 105) with group `git' ... Creating home directory `/home/git' ...
I then switched to the git user account, added a PATH variable and then actually ran the gitolite setup.
sudo su git echo "PATH=$HOME/bin:$PATH" > ~/.bashrc # run gitolite setup, passing a reference to the .pub file we uploaded earlier. In my case this file was in the ubuntu/tmp folder. gl-setup /home/ubuntu/tmp/stefan.pub
Hit enter when asked, and quit the file that opens in VIM with :wq.
I was now able to exit the remote SSH session and could now clone a the remote gitolite-admin repository:
git clone git@ec2-999-239-176-93.compute-1.amazonaws.com:gitolite-admin.git

Amazingly this worked. Please note that I may well have left out a step, or got the order slightly wrong. If needed I could create a screencast sometime (please shout), I think this would make things clearer. Make sure you read the gitolite instructions for how you add users and repos. One mistake I made was to not push my changes to the remote and then was left wondering why my new repos were not showing... Git newbie, I know.

Lastly I downloaded a trial of Tower, a GUI based git client that comes highly recommended - they even have a 25% off offer running right now.

Good luck.

FMS/Wowza/Red5 Broadcaster App For Android

Thu, 12 Jan 2012 14:37:00 +0100

As one of the main contributors to the Red5 project, Paul Gregoire is a well known name in the community. Yesterday he posted a link to an AIR for Android app he built to the Flash Media List and I'm republishing his link here as I think many of you will find this app quite useful.

The Broadcaster Android app allows you to broadcast audio and video to an RTMP server such as Red5, and also works with FMS and Wowza. What's more, the app is free and will run on pretty much any Android device with Android 2.2 and up.

You can download the Broadcaster app from the Android Market.

But apart from being useful, the app also shows that AIR for Android can be quite versatile and allows you to easily publish what is essentially a Flex application to a mobile platform.

Converting And Editing AVCHD (.mts) Files On OSX

Thu, 03 Nov 2011 08:59:00 +0100

A quick note about this article: I chose a slightly misleading title in order to help people find it more easily as most would not know that a search for 'rewrapping .mts files' is what they may be after.

This post is about rewrapping .mts files to make them compatible with QuickTime; it is not about re-encoding or editing .mts files (but rewrapping will make them editable using most common video editing tools).

Many common camcorders and digital cameras - in particular Panasonic and Sony models - produce video files in AVCHD format, with file extension .mts, .m2ts or .m2t. These files often do not play back natively unless you install a third party video player such as VLC.

I was getting a bit annoyed at the fact that quick preview in Finder would not work with .mts files and that the format is generally a bit of a nuisance (iMovie for example won't find any compatible files if you tell it to look in a folder that only contains .mts files). Moreover I was looking to stitch some clips together which in turn was made more difficult by the .mts format.

So what does one do? Of course it is possible to convert the .mts file into a different format (although despite what I read I was unable to open my .mts files in HandBrake), but converting usually means to re-encode which commonly results in a loss of quality. I also did not want to re-encode dozens of videos - it would have taken a considerable amount of time.

I noticed that some tools (including one called Clipwrap) are able to rewrap the .mts file into formats such as .mov which iMovie is happy to open. Rewrapping basically means changing the container format of the file but leaving everything 'inside' intact. This works because AVCHD uses compatible video and audio codecs (video is H.264) which will play fine in QuickTime once rewrapped.
Now tools such as Clipwrap work fine - but its price point is $50 which is a bit steep when all you need to do is rewrap some files. I've also noticed that the free FFMPEG can do the same (and I suspect that Clipwrap may use it under the hood).

So I grabbed myself a FFMPEG binary for OSX (I can't recall which program had it bundled but you can download the binary I used from here) and used the following commandline option to rewrap my files: ./ffmpeg -i test.mts -vcodec copy -acodec aac -ab 512k -strict experimental testoutput.m4v This is an extremely quick process since the file is not being re-encoded. I did so far found one issue related to scrubbing the resulting file; it seems to somehow hang for a few seconds. I'd love to know a fix for this - please leave a comment if you know one.

Apple HTTP Streaming Support Added To OSMF

Fri, 20 May 2011 15:15:00 +0100

Matthew Kaufman, one of the engineers who brought us RTMFP but who is no longer with Adobe, has written and published an AS3 library which adds support for Apple HTTP Streaming (draft-pantos-http-live-streaming) to OSMF. The classes constitute a plugin for OSMF's HTTPNetStream that supports Apple's m3u8 index files and MP2 transport stream media file.

Matthew outlined his efforts on the FlashMedia List:
"I've put the initial commit of this code up at http://apple-http-osmf.googlecode.com with an open-source license (MPL).

No guarantees that it is complete or optimized, but it does appear to play most of the things I've been able to throw at it. Supports H.264 video and AAC audio... adding MP3 audio support is on the list, just need a sample stream to test with and a couple hours to do it."

Thanks Matthew!

VP8 Delivers Better Quality Than H.264 - But You Won't Notice It

Fri, 21 May 2010 10:51:00 +0100

Jan Ozer was quick on the mark to deliver a side-by-side comparison of video encoded with VP8 (the codec which Google open sourced as recently as two days ago) and H.264, the de-facto codec standard for web video and beyond.
You can check out Jan's tests here on streamingmedia.com but in summary it is safe to say that any differences in quality are negligible. What remains to be seen is of course how the same codecs perform across a range of bitrates; maybe VP8 does excel once you throw higher resolution, higher bitrate content at it? Or maybe it will distantiate itself at low bitrates?

But regardless, the mere fact that VP8 is open source now and that it is a serious contender in the codec wars that rage around the web in the past few months is a great thing. Remember that H.264 is a patent encumbered format with a patent pool overseen by an organisation called MPEG-LA, and license fees are payable for certain types of usage. It is the uncertainty about these fees and their possible future rise that give organisations like Mozilla cause for concern - and they are not alone. By open sourcing VP8 Google is obviously prepared to call the bluff of anyone who may claim to hold patents on which the VP8 codec may infringe. Now that the sources are open for anyone to see it is now possible to inspect them, and quite likely sue Google for patent infringement. Of course we don't know yet if that's the case, but I truly hope that this big questionmark will once and for all be cleared up by a court. Hopefully VP8 is either free of patent infringements or Google can strike agreements that shield anyone from being sued if they use the codec. No doubt the web would be a better place if we had a free to use, patent free, high performant codec available for everyone to use with no strings attached. Google is definitely up for it and saying: "Bring it on, whoever you are."

I'm sure Adobe is totally loving this, for various reasons :-)

Another Live Flash Encoder Is Available For Free Download

Thu, 22 Apr 2010 12:36:00 +0100

Disclosure: This post is sponsored by Unreal Streaming Technologies

Unreal Streaming Technologies have recently released version 7 of Unreal Media and Live servers, adding H.264 encoding and the ability to stream to Flash player through the support of the RTMP protocol.

Anyone can download these products from http://www.umediaserver.net/umediaserver/download.html

The Unreal Live Server which serves as a video encoder similar to Adobe FMLE, is completely free of charge, while Unreal Media Server has certain concurrent connections limitation in its free version, and requires licensing.

These streaming tools have a very small download size and small footprint on your system when installed. They have GUI front-end and easy to operate, although they have slightly different concept of operation comparing to other encoders such as Adobe FMLE or Telestream encoder.

The main feature that differentiates Unreal Live Encoder from Adobe FMLE and others is unmanned, automated operation. You configure the system and leave.
Adobe FMLE generally requires a presence of the operator who opens the encoder and presses 'Start Encoding' button. This is good for broadcasting events but may be unacceptable for a system that needs to start broadcasting anytime when somebody wants to watch/listen, such as IPTV, radio, video surveillance, digital signage apps, etc.
Unreal Live Server running as a Windows service will start encoding and streaming when a first viewer sends request for live video by opening a player. It will stop encoding and streaming when a last viewer disconnects.

The latest version of Unreal Media's software adds itself to a small family of free software tools for live Flash streaming.

Unreal Media Server overview:
http://www.umediaserver.net/umediaserver/overview.html

Unreal Media Server architecture:
http://www.umediaserver.net/umediaserver/architecture.html

Flash Media Live Encoder for Mac Now Available!

Tue, 30 Mar 2010 13:11:00 +0100

Hallelujah - Flash Media Live Encoder (FMLE) is now finally available for the Mac platform. The Windows version was also updated, bringing both versions to 3.1.
FMLE now supports both Windows and Mac platforms and both tools are as usual available free of charge.
This latest version also adds supports for the new FMS 3.5 features which include Dynamic Streaming and live DVR. The ability of chunking of locally recorded files will also be very welcome to anyone who has ever had the misfortune to have to try and trim and/or edit FLV files of several GB in size.

The full list of new features is here.

You can download Flash Media Live Encoder 3.1 now.

Flash Media Live Encoder for Mac Announced

Thu, 31 Dec 2009 17:13:00 +0100

Up to now Adobe has been offering the free Adobe Flash Media Live Encoder (FMLE) only as a Windows tool - but this is soon going to change.

The recently announced FMLE 3.1 for Mac OSX will finally bring this essential tool to the Mac platform. This can't come soon enough as it has been requested for a very long time.

FMLE 3.1 for Mac will work with the built-in iSight camera as well as devices such as Firewire cameras and video capture cards to support a broad range of use cases and workflows. Like the Windows version, the Mac version of FMLE 3.1 will be free.

A release date has not yet been announced, but you can register an interest in a private beta by signing up here.

How Does Flash Access 2.0 Relate to FMS?

Thu, 17 Sep 2009 17:42:00 +0100

Jens Loeffler has published a nice little overview on his blog about the differences and similarities between Flash Media Server and Flash Access 2.0 a far as content and DRM mechanisms are concerned.

You can find Jens's article here.

DRM Capabilities To Be Added To Flash Player

Thu, 10 Sep 2009 12:23:00 +0100

Adobe will soon be adding DRM capabilities directly to the Flash Player, a feature that was previously only available through Adobe AIR, Adobe's cross-platform desktop runtime. The new feature was announced at IBC in Amsterdam today.

This is a fairly significant addition to the Flash Platform. While I'm not a fan of DRM, I understand that some content owners are very keen to add (what they perceive as) protection to their assets, and Flash will soon be capable to tick that box. The technology works in a similar fashion to the way in which the Adobe Media Player used to handle content protection: a DRM server called Flash Access 2.0 - which was also announced at IBC and which is basically a renamed release of Adobe Flash Media Rights Management Server (I'm glad that I won't have to type that name again!) - will handle the signing and protection aspects, while Flash Player can soon deal with the decryption and content access mechanisms natively. As mentioned above, this was previously only an option if Adobe AIR was used to build the client. I guess these new features and the demise of Adobe Media Player shows that end users prefer to consume content right inside the browser, and are less keen to install and use yet another video player onto their desktop. Those who prefer to use AIR to deliver their content can still do so as the runtime will also support the content protection features which Flash Access 2.0 promises. Here's the full press release.

Adobe Flash Access 2.0 is planned for commercial availability in the first half of 2010.

Tip for Debugging Remoting Calls in Flex Builder 3

Fri, 21 Aug 2009 16:27:00 +0100

Imagine the times when Flash Builder (Gumbo) was still in beta, times when the Network Monitor was still only a twinkle in the corner of a Flex engineer's eye. Imagine... your current day job, and a Flex 3 project with a bit of ColdFusion powered Remoting (other Remoting gateways are available) thrown in. Which is going wrong. And you need to debug the Remoting calls.
Ok, that doesn't require much imagination, I know. We know that Flash Builder will make our lives somewhat easier when it finally ships, but in the meantime your boss doesn't allow you to run beta software so what does one do besides coming to work for me? And maybe, just maybe, your boss is really mean (unlike myself) and won't even pay out for a license of Charles. So now what?

I present to you . It's a little tag tat does wonders to your Flex Builder 3 Console, and I don't know why it isn't being promoted much more. I've been using it for quite a while, and while I do not recall how I discovered it I am sure it's never too late to tell the world all about it.

So what does TraceTarget do? In a nutshell it outputs a ton of logging information into your Console when you debug your Flex project, and in particular it tells you a lot about the Remoting calls that you are undoubtedly making. Here's an example of my current project, first without TraceTarget added:
[SWF] Users:stefan:Documents:dev:flex:fb3:demo:bin-debug:index.swf - 1,379,424 bytes after decompression result getUserDetails Yeah, nothing basically besides one trace that I out in there to signal the return from CF. Now let's add to the main.mxml file and debug the app again:
[SWF] Users:stefan:Documents:dev:flex:fb3:demo:bin-debug:index.swf - 1,383,632 bytes after decompression '9670B130-68C5-3D75-D251-3D30C4A99F2E' producer set destination to 'ColdFusion'. 'F2AF48B3-896A-FA9D-D77A-3D30C4AF2A71' producer set destination to 'ColdFusion'. '8495032E-7DE7-49B1-124E-3D30C4EB0643' producer set destination to 'ColdFusion'. '8495032E-7DE7-49B1-124E-3D30C4EB0643' producer sending message '17E7B79A-6D4E-21C3-4F67-3D30C4EE538F' 'my-cfamf' channel endpoint set to http://www.demo.com/flex2gateway/ 'my-cfamf' channel settings are: false 'my-cfamf' pinging endpoint. 'my-cfamf' channel is connected. 'my-cfamf' channel sending message: (mx.messaging.messages::RemotingMessage)#0 body = (Array)#1 [0] "abcd" clientId = (null) destination = "ColdFusion" headers = (Object)#2 messageId = "17E7B79A-6D4E-21C3-4F67-3D30C4EE538F" operation = "getRoomDetails" source = "api.v1.private.rooms" timestamp = 0 timeToLive = 0 '8495032E-7DE7-49B1-124E-3D30C4EB0643' producer connected. '8495032E-7DE7-49B1-124E-3D30C4EB0643' producer acknowledge of '17E7B79A-6D4E-21C3-4F67-3D30C4EE538F'. 1 '9670B130-68C5-3D75-D251-3D30C4A99F2E' producer sending message '01B287E5-5EF5-1B40-75EE-3D30C648A026' '9670B130-68C5-3D75-D251-3D30C4A99F2E' producer connected. 'my-cfamf' channel sending message: (mx.messaging.messages::RemotingMessage)#0 body = (Array)#1 clientId = (null) destination = "ColdFusion" headers = (Object)#2 messageId = "01B287E5-5EF5-1B40-75EE-3D30C648A026" operation = "getUserDetails" source = "api.v1.private.users" timestamp = 0 timeToLive = 0 '9670B130-68C5-3D75-D251-3D30C4A99F2E' producer acknowledge of '01B287E5-5EF5-1B40-75EE-3D30C648A026'. result getUserDetails
Pretty cool, right? You can read more about TraceTarget and other logging frameworks here. But if you just want a quick and dirty way to see some more debug info on your Remoting calls then you could do worse than simply sticking the TraceTarget tag into your main.mxml file and save yourself some hassle. It may not be as slick as Charles or the upcoming Network Monitor but it can be handy nevertheless. Have fun coding.

I Know You Know: Google is to Acquire On2 Technologies

Wed, 05 Aug 2009 17:20:00 +0100

I'm sure you've heard the news by now that Google is to acquire On2 Technologies, makers of Flix Standard, Pro, Exporter, Live, Engine and Directshow SDK, plus intellectual property holder of a whole range of video codecs which include VP6, a video codec that Adobe has licensed from On2 and included into the Flash Player (Flash includes a VP6 decoder - the only video encoder in Flash is still Sorenson Spark).
In many respects the VP6 codec played a huge role in the success of Flash video as it quickly became the codec of choice for anyone encoding Flash video for the web.

The Twittersphere is buzzing with chatter about the deal which sees Google handing over around $106.5 million - peanuts in Google's world, a company which is valued at over $150 billion (who really cares about a few billion here or there :).
I'm wondering what this deal means for Flash video and for HTML5. I guess that in the short term, Flash will be unaffected since there is now a clear move towards H.264, and existing license agreements with On2 should remain unaffected too. But what about HTML5? This new standard (which actually isn't one yet) is in real need of a *decent* video codec which is not burdened with royalty fees (as may or may not be the case with H.264).

No surprise that this deal is seen by some (mainly TechCrunch commentators) as yet another Flash (video) killer. Yawn. Others speculate that Google may open source one or more codecs, or make them freely available. That's speculation right now and I could see it go either way, there are many reasons for one or the other (or neither) to happen. We'll see.

Then there are Google's communication tools - a decent video codec is handy for those to say the least, and it's no surprise that Skype is one of the VP7 licensees.

This is a smart move by Google, and I'm actually surprised they haven't done this sooner. The price they paid seems a bargain too, and so Google will soon own the codec technologies which power the majority of web video today. We'll see what Google's plans are from here, but since they do no evil we have nothing to worry about, right? Right? Hello, anyone here?

The full press release can be seen here.

Screen Sharing With JScrCap in Action

Fri, 24 Jul 2009 18:33:00 +0100

Vivek Lakhanpal has posted an interesting entry on his blog where he describes his efforts on getting JScrCap, a Java based Screen Capture kit, up and running with both Flash Media Server and Wowza Media Server.
Vivek describes some of the intricacies when using JScrCap, and I'm quite sure that we will hear more about this tool in the future (I'll try it myself as soon as I get a chance) while the Flash Player keeps lacking native screen sharing support.
One point that Vivek makes is that you can only record files in mp4 format on FMS, not FLV. For more details, check out Vivek's post.