Saturday, 3 November 2012

The Mystery about Sensex

I really wonder why people in India always talk about Sensex without knowing what it is and what is indicates ? Forget about common people, even news channels and financial newspaper always talk about Sensex. Most of the time if you switch to a news channel, they would be talking about Sensex going up or down and predicting India's economic condition by analyzing the Sensex. Sensex is the most tracked and the most talked about index in India. But what is surprising is that it is also least understood index.

What is Sensex ?

Sensex stands for Sensitive Index which is used for Bombay Stock Exchange (BSE). Sensex which is also popularly known as BSE 30 in the financial world is a free-float market-capitalization of 30 well established companies listed on Bombay Stock Exchange. In other words, Sensex tracks price movements of 30 stocks on BSE. During market hours, prices of the index scrips, at which latest trades are executed, are used by the trading system to calculate Sensex  every 15 seconds and disseminate in real time. Sensex makes sense only for Index Fund managers and I don't understand why common man worries about Sensex. Sensex is just a virtual show window of Bombay Stock Market.

Sensex Calculation :

Foremost question which arises is how are these 30 companies selected ?
The top 30 companies with highest free market capitalization are selected. Free market capitalization considers only the shares which are not locked in or the shares which are available for trading. It excludes the shares which are locked by promoters or other strategic investors which can not be traded.
So,
 Free Market Capitalization = Total shares available for trading X Current Market Price

Its clear from the above formula that free market capitalization is dynamic due to the changing market price.
BSE reviews the list of stocks periodically. This essentially means that some stocks are moved out of the Sensex calculation and some are moved in after every review. The 30 stocks selected by the above process are given some weightage  as per its free market capitalization. The base value of Sensex is 100 as of April 1, 1979. The level of the Sensex at any point of time thus reflects the collective free float value of 30 component stocks, relative to the base period. So, a Sensex level of 30000 indicates that the Capitalization of current top 30 stocks is 300 times the Capitalization of top 30 stocks as of the base year.


Myths about Sensex :

1. Sensex mirrors the performance of all the sectors in the Indian ecomony.

The top 30 companies used in the calculation of Sensex do not have nay sectoral quota. These 30 companies mostly include IT companies and Oil & Natural gas companies. Sectors such aviation, gem processing do not have any representation in the Sensex. Hence Sensex can't indicate the performance of all the sectors in the Indian economy.

2. Sensex is an indicator of India's economic growth.

Often Sensex is taken an indicator of India's economic growth. It should be noted that Sensex tracks only the companies which are listed on Bombay Stock Exchange (BSE). Many Indian companies are not listed on BSE. Hence it foolish to consider Sensex as a true indicator of India's economic growth.

3. Sensex tracks performance of all companies listed on BSE.

Sensex tracks only a small set  of 30 companies with the highest Market Capitalization, but there are around 3600 companies listed on BSE. So, even if Sensex rises, many listed individual stocks may have lost value and this scenario has happened many times. In fact on October 29, 2007, The day Sensex touched 20000, 2123 out of 3113 companies declined in value. 

4. Sensex captures price movement of all these 30 companies.

Again wrong. The 30 companies used in the calculation are not given equal weightage. In fact, top 30 companies used is the calculation of Sensex have more than 65% weightage. Thus the rise and fall of these 10 stocks considerably impact the rise and fall of Sensex than the last 20 stocks.

Conclusion :

Sensex is not an unbiased true indicator of top 30 stocks listed on BSE let alone entire stock market and Indian economy. Retail investors should not look at the Sensex and rush into investing in the stock market. A little known fact about Sensex is that its base year has been changed (it was decreased). So, just by decreasing the base year, you can raise the Sensex and this was exactly the motive behind decreasing the base year. The top 30 companies which are used in the Sensex calculation today, were not even present at the time of the base year. So, whats the purpose of this index. In my opinion this base year should be raised at least to the nineties to make some sense.

Saturday, 27 October 2012

Trading Systems - Part 1

Trading Systems are simply set of rules that traders use to determine their entry and exit from a position. In an ideal condition, traders should be like machines which have no emotions. But this is an ideal case and there is practically no emotionless human being. Now the question is what stops us from building a machine which trades.

What is a Trading System ?

A trading system is just a strategy which you develop to trade. You might need different trading systems for different markets because the strategy you use to trade in one market may not be applicable to another market. A trading system consists of 8 parts :
1. A Market Filter
2. Set up conditions
3. Time Frame
4. Entry signal
5. Protective Stop
6. Re-entry Strategy
7. Exit Strategy
8. A position sizing algorithm

A Market Filter : 

A market filter is a way of looking at your market to determine whether this market is appropriate for your system. We can have bullish markets, we can bearish markets, we can have flat markets etc. Your trading system might work well only in one of these market conditions. As a result you need a filter to determine whether your trading system should trade in such market conditions.

Set Up Conditions :

This is your screening criteria. For example, if you trade stocks, there would be 10000+ stocks available that you might decide to invest in at a time.As a result, trading systems employ a series of screening criteria to bring this number down to around 100 stocks. You might have a component in trading system to watch the stock to go down for 7 consecutive days before entry. 

Time Frame :

The first thing when you are developing a trading system is to decide what kind of trader you are. Are you a day trader or a swing trader ? Do you look at charts everyday, every week, every month or every year. How long do you want to hold on to your positions ? This will help determine which time frame you will use to trade.Though you still will be looking at multiple time frames, this will be the main time frame you will use when looking for a trade signal.

Entry Signal :

It is signal that tell you when you might enter a position - either long or short. Entry signal depends on entry rules. The simplest way to generate an entry signal is to employ moving average cross over system. The dual moving average crossover (DMAC) is a simple entry rule. If you employ this rule, a 'buy' entry signal is generated when the shorter term moving average crosses the shorter term moving average. Many traders seem to despise such simple procedures and prefer to use more sophisticated rules.

Protective Stop : 

This is one of the most important components of a trading system. This defines the worst case loss that you would want to experience. Your stop might be some value that would keep you in the stock for a long time(e.g. a 25% drop in the price of the stock) or something that will get you out quickly if the market turns against you(e.g. a 25% drop). Without having proper protective stop, your trading system can incur heavy losses. 

Re-entry Strategy :

Quite Often when you get stopped out of a position, the stock will turn around in the direction that favors your old position. When this happens you might have a perfect chance for profits that is not covered by your original set-up and entry conditions. As a result, you need to think about re-entry criteria. When might you want to get back into a closed out position ? Under what conditions would this be feasible and what criteria would trigger your re-entry ?

Exit Strategy : 

Just as we have a rule which generates a signal which tells us when to enter a market, there is a rule which generates a signal which tells us when to exit a market. Sometimes you might have to exit from a winning position and sometimes you might have to exit from a loosing position. In fact in most of the profitable trading systems, only 25 - 30 % of the trades are profitable. This is the most critical part of the trading system. You must spend a great deal of time on exit strategies. For exits, you have different options. You can either trail your stop, or have a set target and exit  when the price hits that target.


Position Sizing Algorithm : 

A position sizing algorithm just tell you 'how much' and 'how big' of a position to take. Position Sizing algorithm is a key factor in whether or not you stay in the game, or whether your gains are huge or minimal. In its simplest form, it boils down to : how many shares to trade, when to increase, when to decrease, when to take profits and so on. 

Advantages of Trading System :

Emotionless :  A trading system takes all emotions out of trading which is often cited as one of the biggest flaws in individual traders. 

Save Time : A trading system saves a lot of time which a trader would spend on analysis. A trading system can even directly place trading orders without the intervention of a trader.

Disadvantages of a Trading System : 

If a trading system would be without any disadvantages, it would be like a money making machine. But this is not the case. Lets look at the other side of the coin.

Complexity : Trading systems are very complex. You require solid technical expertise in the development phase as well as afterwards to make any changes in strategies. 

Transaction Costs : These are the costs incurred when do a transaction through a trading system. These are more than commission costs.

Time consuming in the development phase : It takes a lot of time to develop a trading system. You have to come up with strategies and then code your strategies. While coding you would have to come up with very efficient code since the data that you use to trade can be huge and you have to process the data efficiently. Also you are directly dealing with money, so it needs rigorous testing. 

Saturday, 20 October 2012

A View on the Transition from Academia To Finance

The other day I was going through an article by Catherine O'Neil from D.E. Shaw group about her transition from academia to finance which I found very interesting. I am sharing that edited article here. First I would like to introduce Catherine.After earning a degree in mathematics from UC Berkeley in 1994, Catherine went to Harvard as a graduate student where she studied number theory and graduated with a Ph.D. in 1999. Then she went to the Massachusetts Institute of Technology as a Moore instructor and then a second postdoc. She started working for DE Shaw group in June of 2007.
Now on to the answers from Catherine:

Why leave Academia ? Why finance ? Why the D.E. Shaw group ?

I found the time scale of academic life frustrating. What started as a moment of insight would take years to get published and disseminated, and that's if you are lucky.
     Finance is a huge and rapidly growing, sexy new field which combines the newest technology with the invention of mathematics to deal with ever growing abundant data. It is the essence of modernity,
and paired with New York City’s infinite energy, I found it extremely attractive. It was really as
simple as that—I didn’t actually know any finance when I decided to apply.
     I first heard of the D. E. Shaw group when Eric Wepsic, my high school math friend, chose to leave Harvard math graduate school to work at this company way back in 1994. Eric would send me emails every year or so, asking if I knew anyone interested in working there, and one day I wrote back and said, “How about me?” The D. E. Shaw group was particularly appealing because it is known for being a well-run company, and since I had decided to try my hand at business, I wanted to start at a good one. Although some people apply to the D. E. Shaw group because it’s known as being pretty academic, I think this is not an appropriate line of reasoning: get a job here if you want to be in business and not academics.

What is D.E. Shaw group like ?


There are a number of groups here and some of them, including my group, focus on the systematic quantitative investment strategies that made the D. E. Shaw group famous, while others
work on more fundamental strategies. I work as a quantitative analyst in a group consisting of about twenty traders, quants, and programmers. A group can be thought of as being similar to an academic department at a university. Groups differ by the type of financial instrument they trade or the means used to approach the trading of a common instrument. Frequently, groups overlap in the type of instrument traded but each group
has its own way of looking at the market.
     In the world of finance, the D. E. Shaw group is special. For example, we have no dress code. Personally I don’t really care about that, but this flexibility has allowed us to attract a number of really exceptional people for whom this is important. More importantly, we are not expected to work insane hours, which is great for me and my young family. When I say not insane, I should mention I work about 9.5 hours a day, five days a week, which is definitely more time than I spent in my office as an academic.

What do you do there ?


On a daily scale, my time in largest to smallest allotments is spent writing code to test models, writing up projects, talking to my manager, talking to other quants in my group, attending or giving a weekly seminar, learning techniques and thinking of new models, and reading business news. As a quant my job is to understand how financial markets work, which is neither purely mathematical nor purely social but which has elements of both. I might come up with an idea using broad economic themes but it is not a model until it is in a testable form involving concrete data. Also, in my group we rotate the responsibility of maintaining the
automatic computer trading system. This is really just a huge program that decides what and when to trade, and it needs constant attention. So for one week in about thirteen, I am on-call basically all the time. As a recent academic, I find this to be the part of my job that is probably the most alien and intimidating, but it is also extremely satisfying to be involved with the nuts and bolts of the operation.
     The transition from academia to finance has meant a shift in my priorities. Unlike when I was in academics, I no longer have to worry about grants, getting papers published and waiting a long time from beginning to end on my projects.  Now working in finance I do worry about the relevance and testability of my ideas, the minute correctness of my code, and of course profit. Leaving an academic career has meant giving up teaching, the students, and the absolute freedom to work on any project I want. On the other hand, finance has provided me with the opportunity to come up with good, new ideas that will be put into effect, be profitable, and for which I will be directly rewarded.

Would I like your job ?


To that question, I would counter with these:
Are you efficient-minded?
Can you sustain focus?
Are you flexible about the field to which you apply your quantitative talents?
Do you like to understand how systems work as well as the theory behind them?
Do you enjoy mastering new skills?
Do you appreciate the existence of a “bottom line”, a way to quantitatively measure the success of your projects and your ideas?
Are you articulate?
Are you good at following through and finishing projects?

     Notice I didn’t ask if you are particularly informed about finance or money per se, because honestly I wasn’t when I decided to enter finance. I don’t think it was a disadvantage, and now I really enjoy finance and find myself reading finance books instead of fiction. I had also never coded, but now I really enjoy coding. Both of those are skills that anyone with focus, intelligence, and flexibility can master and enjoy. For me and for many of my colleagues it is intrinsically satisfying to be in a collaborative atmosphere as part of
a functional, productive, and hard-working team with clear goals.

What does the D.E. Shaw group look for ?


The D. E. Shaw group hires people of extraordinary ability. Quant candidates typically come from math, physics, or computer science backgrounds and often have Ph.D.s. This is not to say having a Ph. D. is a requirement, but certainly being capable and smart enough to have a Ph.D. is. What we are really looking for is new ideas, and so our target is the creative, careful thinker. We look for evidence of such talents in the forms of published original papers as well as original personal projects or specialized hobbies. We typically do give brainteasers in interviews, but it is not true that only people who are insanely quick at brainteasers are seriously considered. I do not consider myself all that quick, for example, but I am methodical, articulate, and I don’t make huge mistakes.

Is Finance a good place for women ?


Working in finance is different from working in academics in that there is no tenure. However, I would recommend thinking about the concept of employability over job security. Although a given company may not last forever, the finance industry is here to stay, and there is always a need for quantitatively strong people. If you find yourself out of a job, but you have real skills and knowledge, chances are you will find another job quickly.
     Partly because of this consideration, the D. E. Shaw group tries very hard to get great people and keep them. The turnover is low, partly due to our selectiveness in hiring only the very best people, and partly because people feel valued and don’t want to leave. In fact some people have been known to retire early, but soon change their minds and return. There is little burn-out because the hours and conditions are reasonable.

What is corporate culture like ?


It is really different. People are both more competitive and more collaborative. They are more competitive in the sense that there’s lots of money involved, and therefore getting credit for an idea that makes money is a direct channel to getting paid better. At the same time, everyone relies on their colleagues to keep the whole thing running and so it is imperative that we work as a team. It’s an intense, challenging, and exciting environment to work in.
     About the money: many mathematicians who talk to me about moving to finance are genuinely worried about the potentially corruptive power of money. I take that fear very seriously, and I think I probably would have applied to the D. E. Shaw group earlier if I hadn’t experienced it myself.
     Several factors have helped me come to terms with this concern. First, it is really expensive to live in New York, especially with kids. So actually as a new quant, you are not all that rich, even though you are making more than almost all academics. However, it is clear that if you stay in finance for long enough, and are successful, you do become rich. Even so, I do not find my colleagues to be particularly acquisitive, and indeed some of them are known to support progressive causes and charities such as the Robin Hood Foundation, and I’m sure many of them quietly do so as well. In fact it is a stated goal of the D. E. Shaw group to foster an ethical work environment and to do what’s right.
     I think one can resist being corrupted by money by keeping a perspective and maintaining personal boundaries. I personally give a certain amount of my paycheck to my favorite grass-roots charity. I thereby see working here as a fantastic and rare opportunity to have a great job and to improve the world in some small way simultaneously.





Saturday, 6 October 2012

Financial Information Exchange (FIX)

What is FIX ?

FIX was originally called Fidelity Information Exchange, but now FIX stands for Financial Information Exchange. FIX is an industry driven standard to communicate trading information electronically between brokers, buyer institutions, seller institutions and markets. FIX is platform independent, so it works with various types of computers and communication systems. FIX has been developed through the collaboration of banks, institutional investors, brokers and exchanges. These market participants wanted a common language for automated trading of financial instruments.

Brief History

FIX was first developed in 1992 by Robert Bob for equity trading between Fidelity investments and Salomon Brothers. Hence it was first called Fidelity Information Exchange. FIX has now become the de facto messaging standard for pre-trade and trade communications in the global equity market. FIX has now entered post trade space as well as foreign exchange, fixed income and derivative markets. FIX Protocol Ltd. is the company established for the purpose of ownership and maintenance of the specification. FIX is gaining increased attention within the exchange community as over three quarters of all exchanges surveyed supported a FIX interface, with the majority handling over 25% of their total tarding volume via FIX.

FIX Connectivity Diagram

Financial Information eXchange System Connectivity Diagram.svg
source : http://en.wikipedia.org/w/index.php?title=File:Financial_Information_eXchange_System_Connectivity_Diagram.svg&page=1

FIX Messages

FIX session is layered on TCP. FIX messages are formed from a number of fields, each field is a tag value pairing that is separated from the next field by a delimiter ASCII 01. The tag is a string representation of a integer that indicates the meaning of the field. The value is an array of bytes that hold a specific meaning for the particular tag. FIX protocol defines a set of fields that make up a message. Some of these fields are mandatory and some are optional. The ordering of the fields is unimportant. A FIX message is composed of a header, a body and a trailer. Following is an example of FIX message:

Sending: [
 BeginString 'FIX.4.2'
BodyLength '200'
MsgType 'D' "Order - Single"
MsgSeqNum '81'
SenderCompID 'DSCWE10'
SenderSubID 's1'
SendingTime '20100914-13:34:27.643'
TargetCompID 'BCAPSUBM'
PossResend 'N'
Account 'DESHVALI'
ClOrdID 's1bs6940_20100914'
Currency 'USD‘
OrderQty '100'
OrdType '2' "Limit"
Price '129.45'
Side '1' "Buy"
Symbol 'IBM'
TimeInForce '0' "Day"
TransactTime '20100914-13:34:27'
ExDestination 'N'
MaxFloor '100'
SecurityType 'CS' "Common Stock"
CheckSum '161'
]

Recovery Handling : 

FIX has an optimistic model for recovery. It does not has per message acknowledgement. Instead the receiver monitors sequence number and detects gaps. Sequence number increases monotonically and is generally reset per day. There are serious errors is sequence number is decreased.

Saturday, 25 August 2012

Thread Locals in Java

Thread Locals

Thread local  is a scope just as we have static scoped variables which belong to a class or instance scoped variables which belong to an Object. Thread local variables belong to a thread. Each thread would have its own thread local variable. So, threads can't modify each other's thread local variables. Thread variables are a sort of global variables which are restricted to a thread.

When do we use Thread Local variables ?

By default, data is shared between threads. You can refer to my previous post to get an idea about what is shared among threads. You can use Thread Local variables when you want each thread to have its own copy of something. One very important use cases of thread locals is when we have an object that is not thread safe, but we want to avoid synchronizing access to that object. It would be more clear from an example.

Suppose I have a requirement to use use Java Calendar object in my code. Since Java Calendar is not thread safe, we can either have Calendar object as an instance variable or have it as a class variable and provide synchronization to access it. The first method can't be used in most of the production codes because Calendar object creation is an expensive operation. And still if we have 2 threads of the same process accessing the variable, we need synchronization. The second method looks good since we would have only one object of Calendar class, but we would have to take care of synchronization. 

If we don't want to bother about synchronization, we can go for thread local variables in which case, each thread would be given its own local copy of the thread local variable. Another alternative to thread locals or synchronization is to make the variable a local variable. Local variables are always thread safe. But in our case, since Calendar object creation is an expensive operation, it is not recommended to use local variable for Calendar. Since each time the method is called, a Calendar object would be created which is an expensive operation, it would slow down the.

Another very important use case of Thread Local variables is when we want to associate state with a thread. Many frameworks use ThreadLocals to maintain some context related to the current thread. Web applications might store information about the current request and current session in thread local variables so that the application has easy access to them without passing them as parameters every time. Let me explain this with a scenario.

Lets say, we have a Servlet which calls  some methods. You have a requirement to generate a unique transaction ID for each request you receive and pass this transaction ID to the business methods for processing. One way is to generate a unique transaction ID each time the servlet receive a request and pass this trasaction ID to the methods which require it. But this doesn't look good, since passing of transaction ID to all methods which require it is redundant and unnecessary. Instead, we can use thread local variable to store the transaction ID. Every method which requires transaction ID can access it through the thread local variable. The servlet might be receiving many requests, but each request is processed in a separate thread. So, each transaction ID would be local to a thread and would be accessible all through the thread's execution which is what I mean when I say that Thread Local variables are global.

Usage of Thread Local variables in Java

Java provide a class named ThreadLocal by which you can set and get Thread Local variables.
Typically Thread Local variables are static fields in classes. The code below shows you how to create a Thread Local variable.

Problems with Thread Locals

Thread Locals also comes up with many problems and you have to be careful while using thread locals. Thread Locals can lead to classloading leaks. Thread Locals are very dangerous when it comes to long running applications and garbage collection. Let me explain this point a little bit.

If you use thread locals to store some object instance, there is a high risk that the object stored in thread local is never collected by garbage collector when your application runs inside WebLogic Server. This is because WebLogic server maintains a pool of working threads even when the class that created it is garbage collected. So, if you do not clean up when you are done, any references that it holds as part of the webapp deployment will remain in the heap and would never be garbage collected. This problem can be solved through the proper use of Weak References with Thread Locals. 

Sunday, 19 August 2012

Processes andThreads

In this post I would be discussing about processes and threads.

Process

A process is an instance of a program that is being executed. A process consumes the resources of an operating system. Since there are many processes running at a time, how does the OS manages its resources ? To manage processes, an operating system has a process table. A process table is a data structure which includes the following information:
  • Process ID
  • Process Owner
  • Process priority
  • Pointer to the executable code of the process
  • Parent Process
  • Environment variables
  • Process state
A process can have many threads of execution. By default, any running program has a single thread of execution. A process has a unique address space which is generally not shared with any other process, except during inter process communication, the operating system can relax this condition.

Threads

 A thread is a smallest unit of execution that can be scheduled by an OS. A thread is called the light weight process because thread creation can be 10-100 times faster than a process creation. This is because threads share address space unlike processes which do not share address space. Here I am talking about the threads of the same process. Threads of different processes, of course do not share address space. The main reason for having threads is that in many applications, many activities are going on at the same time. Some of these activities may block from time to time. By decomposing such an application into multiple threads, we increase performance. Threads yield no performance gain when all of them are CPU bound, but when there is substantial amount of I/O as well as computing. Having threads, allows the activities to overlap, this speeding up the application. Threads also allow parallel execution on a multiprocessor system. In this case, the programmer needs to be careful to avoid race condition.

A thread has the following information:
  • Thread ID
  • Program Counter
  • Register Set
  • Stack
So what does the thread share with other threads of the same process?
  • Code section
  • Data section
  • OS resources
Lets talk about the advantages of threads.

Thread Advantages:

  1. Thread creation and destruction is faster than process creation and destruction. 
  2. A thread has lower context switching overhead than a process. This is because a thread has a lesser context than a process because threads share address space. Remember here I am talking about thread of the same process.
  3. Information sharing between threads is easier and has less overhead because threads share address space. So, data produced by one thread is immediately available to all other threads of the same process.

Thread Disadvantages:

Since global variables are shared between threads, inadvertent modification of shared variables can be disastrous. It calls for concurrency control measure which have their own complications.

Types of Thread Implementations:

There are 3 types of thread implementations:
  1. User Level Threads
  2. Kernel Level Threads
  3. Hybrid implementation

User Level Threads:

The type of thread implementation puts the thread package entirely in user space. The kernel is not aware of threads. The kernel just knows that it is managing single threaded processes. Like an OS maintains a process table, a process maintains a thread table which does the same job as process table does for operating system. Each process has its own private thread table. In this implementation, when a thread wants to go to the blocked state, it notifies the run time system. The run time system saves the thread state in the thread table and looks for a ready thread in the thread table to run. We see that in case of user level thread implementation, we don't trap to the kernel in case of thread switching. This is at least an order of magnitude faster than trapping to the kernel in case of kernel level thread implementation.

The main problem with this type of thread implementation is that if by chance any user-level thread is blocked in the kernel, all threads of that process are blocked. Another problem with use-level threads is that we don't take advantage of multiprocessing since the kernel is not aware of any threads.

Kernel Level Threads:

In this type of thread implementation, any thread in a process would be mapped to a kernel level thread. Switching between threads in this case requires kernel mode switch which is expensive. When a thread blocks, the kernel, at its option, can run either another thread from the same process or a thread from a different process. With user level threads, the run time system keeps running threads from one process until the kernel takes the CPU away from it.

Saturday, 4 August 2012

Website Parsing

Today I would be writing about a website parser which I wrote. In this post, I would show you how to parse www.cricbuzz.com website. But the logic is nearly the same for other websites also. You can play around by changing the logic according to your needs. I have used Python in my code, so you need to know Python to follow this post.

So lets begin parsing cricbuzz site. First go to that website. Go to the page that you want to parse. Suppose I want to parse the ongoing England Vs South Africa test match. Go to the Full scorecard page of cricbuzz as shown below:


.
 If you are using Google Chrome browser, press Shift + Ctrl + J to go into the developer mode. You would see a new split window having some tabs as shown below:




Then click on the Network Tab:




Then click on scorecard.json which is highlighted in the above picture.



We see that this site uses JSON which is a light weight data interchange format to send the data. JSON is easy for machines to parse and generate. It is based on Javascript Programming Language. Now you can use your logic to parse the site. I will be using Python's Json package to parse the Json content. Lets start with the code. First import json package. We would need the URL of the JSON page to begin parsing, so get the url by right clicking on scorecard.json.  You can check that URL by pasting in your web browser. You should see a page like this :




We need to get the data from this URL to begin parsing. We can use urllib2 package for this task. The following statement would get the whole data in result string, where the URL is the copied URL:

result = json.load(urllib2.urlopen(URL))


The logic I have used is that if the score changes after 20 mins, it would send an email to the person. To handle the email part, we have to use smtp package.

So, here is the complete code:




The code I have used has very little practicality, but the idea was to make the concept clear. If the idea was clear, you can play around with the logic.  :)