Twitter Scalable Archtecture

Three is a talk Timelines at Scale by Raffi Krikorian @raffi.

The take away:

Outliers, those with huge follower lists, are becoming a common case. Sending a tweet from a user with a lot of followers, that is with a large fanout, can be slow. Twitter tries to

Why It is not Good to apply for a Milage Credit Card?

I did a thorough research on whether or not to apply for a milage credit card. Here is the final analysis:

Million Query Track - Knowledgebase Acceleration

Million Query Track http://ir.cis.udel.edu/million/

The Million Query Track serves two purposes.  First, it is an exploration of ad-hoc retrieval on a large collection of documents.  Second, it investigates questions of system evaluation, particularly whether it is better to evaluate using many shallow judgments or fewer thorough judgments and whether small sets of judgments are reusable. Participants in this track will run up to 40,000 queries against a large collection of web documents at least once. These queries will be classified by assessors as "precision-oriented" or "recall-oriented". Participants can, if so motivated, try to determine what the query class is and choose ranking algorithms specialized for each class.

The task is a standard ad hoc retrieval task, with the added feature that queries will be classified by hardness and by whether the user's intent is to find as much information about the topic as possible ("recall-oriented"), or to find one or a few highly-relevant documents about one particular aspect ("precision-oriented").
Here are the query types:

Wresting with java on running processes

Take away of a couple of hours of wrestling with java: Java is stupid in taking care of running external processes and loses track. I executed around two million system processes, to get there on big data processing.

(pool-5-thread-5) java.lang.OutOfMemoryError: unable to create new native thread   

Also generated error was: not enough resources to run the next processes.

The fix:

Myth For Loop O(n) but O(n^2)

What's the time complexity of

for(int i=0; i < linkedlist.size(); i++){
     System.out.println(linkedlist.get(i));
} 

Contrary to the notion of that a simple for loop should be O(n), the above example is O(n^2). The reason being

How to(Steps)Getting Started in Cloud: Amazon EC2

Ok,

1) Create an account on aws.amazon.com, you will need a credit card to enter but you won't be charged that's for security purposes.
2) Hover your mouse over My Account / Console, click AWS Management Console. This will take you to https://console.aws.amazon.com/
3) Click on EC2, This will take you to https://console.aws.amazon.com/ec2/
4) On the left pane of the screen click on Key Pairs. Click Create Key Pair. Enter a name for your key click create. Your key -pair will be created and your private key will start downloading. Save it at a  safe place.
5) Get back to EC2 Console https://console.aws.amazon.com/ec2   Click on Launch Instance, this will take you to https://console.aws.amazon.com/ec2/home?region=us-west-2#startWizards=true  
6) Quick Launch Wizard > Choose a Launch Configuration, scroll down

Make Java Memory Efficient

1. in getting substrings from a BIG string make a new string like here:
String mySubstring = new String(orig.split(";")[i]); 

OR new String(offset + beginIndex, endIndex - beginIndex, value);

Taken from here and here.

If you don't mind sacrificing a little accuracy vs more memory efficeiency use bloomfilter instead of hashmap.
Taken from here and here.

Use efficient designs: stuff like Flyweight Pattern! مگس وزن
use as many abstractions as you can, don't worry about it, it would pay off

I didn't know there is a method called intern for String! Note that

How to master your tmux like a BOSS! Generate tmux panes

Here is a killer point! Whatever you do with your tmux call  

$ tmux source-file .tmux.conf   

So I was monitoring several processes on a server from my laptop and whenever I connected to an ssh and lost my connection my tmux would freeze and I had to rebuild the whole structure:

First row:

Why Java Calculates Date Time Wrong

I needed to iterate through a date-time range and observed some strange behavior from Java. So I want to iterate hour by hour from 2011-10-05-00 until  2013-02-13-23. The code I was expecting was

DateFormat format = new SimpleDateFormat("yyyy-MM-dd HH");
  
Calendar c = new GregorianCalendar(2011, 10, 5, 0, 0);  
Calendar cEnd = new GregorianCalendar(2013, 2, 14, 00, 0);
  
while (c.getTime().before(cEnd.getTime())) {
 System.out.print(format.format(c.getTime()) + "[");
 System.out.println(c.getTimeInMillis() + "](" + cEnd.getTimeInMillis() + ")");
 c.add(Calendar.HOUR, 1);
}

For some strange reason that I haven't figured out yet this will iterate from 2011-10-05-00 until 2013-03-13 23 (one month later than I'd expected!).

Parse the string first!

Vim colorscheme Mustang

I was having some issues setting up a custom color scheme for vim.

Download a colorsheme e.g. mustang from here move it to ~/.vim/colors
In your ~/.vimrc add line
       colorscheme mustang
I was getting

  Error detected while processing /home/morteza/.vimrc:
  line    2:
  E185: Cannot find color scheme mustang
  Press ENTER or type command to continue

To fix this rename

What if your connection drops while you are in the middle of SSH?

If your connection drops in the middle of an ssh session, the TTL of your session will terminate your job running via that session after a while. To avoid

Java CPU/Memory Heap Usage Monitoring A.K.A. Java Profiling

Java Profiling: One of the important tools needed when working with java is the use of profiling tools to tell you about the memory consumption and CPU usage of each method. VisualVM is a free and easy to use tool. Java profiling will tell you how heap is being used, Garbage Collector operations, threads activities, CPU time of each method, which object is taking most of memory and etc.

To set it up