Disable ubuntu update manager, timer from command line

On command lines:

sudo systemctl disable apt-daily.timer
sudo systemctl disable apt-daily-upgrade.timer

then edit 2 files:

sudo vi /etc/apt/apt.conf.d/10periodic
sudo vi /etc/apt/apt.conf.d/20auto-upgrades

In both files, change "1" to "0".

Creating Time Lapsed video from dashcam videos

Time lapsed video are usually create from photos.  Here's how to create one from dashcam videos.
First move all videos into a single folder, then list files and redirect output into a text file.  On linux, it's `ls > files.txt`, on Windows, it's `dir /B > files.txt`.

Then edit this file.  This file should list videos in the order of timestamp, if not, adjust them manually. Remove any files that's not video or any videos that you don't want to be included in the final video.  Put 'file ' at the beginning of every line.  (In Vim, you can do this by :%s/^/file /g)

Then create a raw video by select one frame in every 30 frames from all the videos in the file:

ffmpeg -f concat -i files.txt -vf "select=not(mod(n\,30)),setpts=N/(FRAME_RATE*TB)" -vcodec rawvideo -pix_fmt yuv420p -an raw.yuv

Because this is uncompressed raw video, its size will be huge, probably tens of GigBytes. You probably don't want to play it on your computer, let alone uploaded to youtube.  To make it playable, we need to compress it to mp4 format:

ffmpeg -f rawvideo -pix_fmt yuv420p -s:v 1920x1080 -i raw.yuv -vcodec libx264 video.mp4

 You can see an example of the time lapsed video here:

Tornado vs Starlette in 2019

For the last 10 years I have been using mainly  Tornado as my web framework of choice.  And I mostly use it synchronously.  Only when dealing with uploading files did i use some asynchronous tornado features.   Most of my apps have database backends.  With database involved, it's not worth it to convert my code to asynchronous mode.  Tornado has been worked well for me I have no intention of stopping using it in the near future.  But I do want to keep my eyes open for new tool that enables me to build fast app fast.

In the last few years, there are several new async web frameworks.  One of them is Starlette, I decided to evaluate it.  The first step is to compare it basic performance with Tornado.

Starlette, using uvicorn, claims to be one of the fastest python web framework.  I copied the codes directly from its website:

from starlette.responses import PlainTextResponse

async def app(scope, receive, send):
    assert scope['type'] == 'http'
    response = PlainTextResponse('Hello, world!')
    await response(scope, receive, send)

I run it with "uvicorn example:app", then measure rps using 'ab':

ab -n 1000 -c 10 127.0.0.1:8000/

 The result is on average 4200 requests per second.

I then run basic Tornado app, again with code copied directly from tornado website:

import tornado.ioloop
import tornado.web

class MainHandler(tornado.web.RequestHandler):
    def get(self):
        self.write("Hello, world")

def make_app():
    return tornado.web.Application([
        (r"/", MainHandler),
    ])

if __name__ == "__main__":
    app = make_app()
    app.listen(8888)
    tornado.ioloop.IOLoop.current().start()

 Using the same measurement, the ab result is a average 1900 requests/second.

So it looks like Starlette/uvicorn is more than 2 times faster than Tornado.   However, the code on Tornado website is actually not the best for production use.  Just replace "app.listen(8888)" the second to last line with following:

server = HTTPServer(app)
server.bind(8888)
server.start(0)

(and put "from tornado.httpserver import HTTPServer" at the beginning of the file),  its performance increases to 4700 requests/second, actually faster than Starlette.

The code change to Tornado make the app start 4 processes instead of 1.  Of course we can also do the similar for the Starlette app:

uvicorn --workers 4 example:app

The result now is 7500 requests/second, surpassing multi-process Tornado again.

These are superficial results that don't mean much.  But it did make me appreciate the works that got into Tornado that keep it performant in these years.

Result Summary:

• Tornado (default): 1900 rps
• Startlette/uvicorn (default): 4200 rps
• Tornado (4 workers): 4700 rps
• Starlette/uvicorn (4 workers): 7500 rps

DrJava Font Size Problem

I'm helping my son learning Java. One of the first things to do when learning Java is picking an IDE. I like DrJava for its simplicity and small size. However, when I first run DrJava, its font size is way too small on my 4K monitor.

There are 2 ways to fix this - the easy way and the hard way.

The easy way first.
Click Edit:Preferences, then click "Display Options", change "Look and Feel" to the one that ends with "Plastic3DLookandFeel".  Do this on Windows only, for Mac and Linux, default "Look and Feel" is fine.
Click "Font" under "Display Options", change all fonts to double the original size:

Press "OK", then close DrJava then open it again.  Now the fonts should be big enough to read.

(The changes you make are actually saved in a configuration file .drjava in your home directory,  you can edit the file directly for changes, but it's not recommended)

Now the hard way.
For the "easy way", we just changed DrJava's default configuration preferences.  The hard way is to compile a DrJava program with these changes already made in the source code so we don't have to change preferences.

Clone the DrJava repository on github: https://github.com/DrJavaAtRice/drjava

Try to compile it first:

cd drjava/drjava
ant jar

You must have ant already installed.  If it's compiled OK, try run it:
java -jar drjava.jar
(You can just double click drjava.jar file too)

Open src/edu/rice/cs/drjava/config/OptionConstants.java for editing:
Change "Monaco-12" to "Monaco-24"; change "Monospaced-12" to "Monospaced-24";
chanage "dialog-10" to "dialog-20"; change "dialog-12" to "dialog-24".

Save the file then recompile (ant jar).  Now DrJava is "pre-configured" with big fonts.

You might say this "hard way" is pointless.  Why on earth would anyone want to do this? I agree.  But just maybe one wants to provide a "pre-configured" copy of DrJava to his students.  I just show one way to do it.  Another way is to write a .drjava configuration file upon installation.  This is way more complex. Besides, I'm using a standalone jar file, there's no installation to speak of.

Bugzilla "Cannot determine local time zone" problem

I had this problem today and was pulling my hairs out trying to find a solution.
Below is the only fix that worked for me:
In 'lib/DateTime/TimeZone/Local.pm', find 'sub TimeZone {', right after it, put this next line:

return DateTime::TimeZone->new(name => 'Asia/Shanghai');

(Replace 'Asia/Shanghai' with your own timezone.)
Basically all the rest of this sub has no effect now.

Reload the error page, if it show another error says: "List::Util version 1.31 required...", open 'lib/List/AllUtils.pm', goto line 10, or find the line:

use List::Util 1.31 ();

Just delete '1.31'.

After that, it should work.  I'm using Bugzilla 4.4.9.

Cloning HDD to a smaller SSD for a Windows 8 Laptop

My win8 Dell Laptop has a 500GB HDD, I bought a 480GB SSD for hard disk replacement.  Unfortunately, the 500GB HDD's partitions span the whole disk, the exact layout copying wasn't possible.

I formatted the SSD to a GPT disk, then use various cloning softwares to do the cloning.  I tried Macrium Reflect Free, EaseUS Partition Master, Minitool Partition Wizard. None of them get me to Windows startup screen.

The last one I used is Minitool Partition Wizard. During boot, it said can't find OS.  So I tried changing different bios settings out of desperation and hoped one would work.  It did.

The setting is "disable virtualization".  I don't know why it worked, it doesn't make any sense, I'm just glad it did.

So if you have problem migrating HDD to SSD, this might be something you can try.

Re-sizing a Virtualbox disk image in Linux

One of my VirtualBox virtual machine had its disk full, I thought a simple resize should fix the problem. And I went:

vboxmanage modifyhd u140464-02-disk1.vmdk --resize 30000
0%...
Progress state: VBOX_E_NOT_SUPPORTED
VBoxManage: error: Resize hard disk operation for this format is not implemented yet!

What?!
Turned out "modifyhd" only support .vdi, not .vmdk. OK I will just use .vdi then:

vboxmanage clonehd u140464-02-disk1.vmdk u140464-02-disk1.vdi --format vdi
vboxmanage modifyhd u140464-02-disk1.vdi --resize 30000

Then detach vmdk disk from the VM and attach the vdi.
But after I boot it up, the disk was still full. The disk size wasn't changed. After some googling I found I need to install gparted and use it to grow the partition, and gparted has to be run at boot. Too much hassle.  All I want was some additional disk space. So I did it my way.
(Inside the VM)

sudo fdisk /dev/sda

then enter p, it showed:

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048      499711      248832   83  Linux
/dev/sda2          501758    33552383    16525313    5  Extended
/dev/sda5          501760    33552383    16525312   8e  Linux LVM

Enter n to create a new partition, press enter 2 times to accept default partition size. For some reason, there was a stray segment, so I have to enter n again to create a new partition. 2 partitions were created, after a p, my partition table looked like this:

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048      499711      248832   83  Linux
/dev/sda2          501758    33552383    16525313    5  Extended
/dev/sda3          499712      501757        1023   83  Linux
/dev/sda4        33552384    61439999    13943808   83  Linux
/dev/sda5          501760    33552383    16525312   8e  Linux LVM

sda3 is useless, so I deleted it by a 'd' followed by a '3'. sda4 is the one I will use. Enter 'w' to write to disk. Then reboot the VM.

After reboot, do a:

sudo mkfs.ext4 /dev/sda4

My /opt was not used, so I just mount sda4 as /opt

sudo mount /dev/sda4 /opt

Add "/dev/sda4 /opt ext4 defaults 0 2" to the end of /etc/fstab so it was mounted automatically at boot.

In the end, I should have just add another virtual disk without bothering resizing the current one.

fixing VPN problem in Ubuntu

I set up VPN in Ubuntu 14.04, but when I tried to turn it on in NetworkManager, I kept getting errors like "No suitable device found", "Could not find source connection" in /var/log/syslog.

The problem turned out to be that my ethernet connection was not managed by NetworkManager. "nmcli c" didn't show my wired connection.

The solution:
Edit /etc/NetworkManager/NetworkManager.conf, in section [ifupdown], change "managed=false" to "managed=true", then do a:

"sudo service network-manager restart"

Now both vpn and ethernet connections are managed by NetworkManager, turning on VPN just works.

Programming Minecraft with Python: Introducing JuicyRaspberryPie

I developed a Bukkit plugin called JuicyRaspberryPie, that let you add commands to Minecraft with Python programming.

It's largely based on RaspberryJuice plugin, I added a Python command server and also changed the Java code a little bit to let the plugin talk to the command server.

The github page has more detail.

Here's a video demo of the plugin. (Fast forward to 15 minutes for python examples, before that is just about how to download and install the plugin, which is pretty boring)

Solving windows installer error 2503 2502 problem

I got a new laptop and tried to install python on it. But it gave me 2503 2502 error.  I had installed python on other windows 8 machines and never had this problem.

2503/2502 problem is caused by insufficient permission, which should be fixed easily, just right-click and then "run as Administrator", right?  Wrong! There's no "Run as Administrator" option available for MSI (windows installer) file.

Now the solution: open explorer, find your way to C:\Windows\System32, locate cmd.exe, right click it, then "Run as Administrator".  (If that's too complicated, try this: press "windows+Q", type "cmd", right click "Command Prompt", then "run as administrator").

Now cd to the directory where download the MSI file, then type the filename and enter. Install as usual.