Ivan Idris Blog

July 25, 2010

Grails Finance 0.3

Filed under: programming — Tags: , , — admin @ 8:59 pm

So now that I have historical data in my database, it’s time to have a good look at it. One popular way is to represent the data in a chart. Creating graphs on your own from scratch is a lot of work. Luckily, there are a lot of freely available charting API’s out there. For some of those API’s Grails plugins exist.

Google Charts

Google Charts generates an image from an URL with parameters. The Google Chart Grails plugin has a set of tags with, which you can specify what kind of chart you want. Install it

1

grails install-plugin google-chart

The testing procedure on the plugin information page did not work for me and it seems that the plugin does not support all the Google chart types. Other than that, everything seems to work fine.

Controller

I added two methods in the controller. The first one, gets historical end of day data sorted by date, for an instrument and a field. For instance, DIA and the high price field. The second method calls the first one for two different securities and the same field in order to compare the data in a chart. This method also produces a list of x axis labels.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

   def historicalChart = {
     if(params.fieldVar == null) {
         return [instruments : getGoogleInstruments()]
      } else {
         def instrument = Instrument.findBySymbol(params.select1)
         def instrument2 = Instrument.findBySymbol(params.select2)
		 def field = Field.findByName(params.fieldVar)
		 def firstData = getDataForInstrument(instrument, field)
		 def secondData = getDataForInstrument(instrument2, field)
 
		 def data1 = [:]
		 def data2 = [:]
		 def xLabels = []
 
		 for(def added : firstData?.keySet()){
			if( secondData?.keySet()?.contains(added)) {
				def shortKey = added.toString("MMM yy")
 
				if(!xLabels.contains(shortKey)) {
					xLabels << shortKey	
				}
 
				data1[added] = Double.valueOf(firstData?.get(added))
				data2[added] = Double.valueOf(secondData?.get(added))
			}
		 }
 
         return [instruments : getGoogleInstruments(),
			 firstData : data1,
			 secondData : data2,
			 xLabels : xLabels]
      }
   }
 
   def getDataForInstrument(instrument, field) {
         def fieldVals = FieldValue.findAllByInstrumentAndField(instrument, field)
		 def data = [:]
 
		 for(int i = 0; i < fieldVals.size(); i++) {
			data[fieldVals[i].added] = fieldVals[i].val 
		 }
 
		 return data.sort { it.key }
   }

The view

The view for the chart consists of one GSP page. I put some logic in it, that possibly could have been better placed in the controller. The page consists roughly of two parts. The top part is a form, which lets you select two instruments and a field, such as a price to plot. The bottom part displays a line chart with a special tag.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

<html>
   <head>
      <title>Chart Historical data</title>
   </head>
   <body>
      <table>
         <tr>
            <g:form>
            <td>
            		<g:select name="select1" from="${instruments.symbol}" value="${params.select1}"/>
            </td> 
            <td>
            		<g:select name="select2" from="${instruments.symbol}" value="${params.select2}"/>
            	<td>
            <td>
               <g:select name="fieldVar" from="${['Open', 'High', 'Low', 'Close', 'Volume']}" value="${params.fieldVar}"/>
            </td>
            <td>
               <g:actionSubmit name="doChart" action="historicalChart" value="Chart"/>
            </td>
            </g:form></td>
         </tr>
      </table>
 
       <g:if test="${firstData != null}">
       		<% 
			   def firstMax = firstData?.values()?.max()
			   def secondMax = secondData?.values()?.max()
			   def yMax = Math.max(firstMax, secondMax)
 
			   def priceData = [firstData?.values()?.asList(), 
				   secondData?.values()?.asList()]
			%>
	        <g:lineChart type="lc" title="End of day (${params.fieldVar}) comparison"
				size="${[600,200]}" 
				colors="${['FF0000','0000FF']}" 
				axes="x,y" 
				lineStyles="${[[2,2,2],[2,8,4]]}" 
				legend="${[ params.select1, params.select2 ]}" 
				gridLines="${100.0/11},25" 
				axesLabels="${[0:xLabels,1:[0,yMax/2,yMax]] }"
				data="${priceData}"
				dataType="simple"  />
		</g:if> 
   </body>
</html>

Result

The result of this little science project is a table of charts. I plotted DIA, a Dow Jones Industrial Average tracker, SPY, a Standard and Poors 500 tracker and GLD, a Gold Trust tracker against each other for OHLC and volume parameters. The underlying indices of the first two trackers have similarities, therefore I expect, that the trackers wil be also linked in some way. GLD was added for good measure, because as 9 out of 10 scientists will, tell you, gold is very shiny :) .

DIA x SPY DIA x GLD SPY x GLD
Open DIA SPY Open DIA GLD Open SPY GLD Open
High DIA SPY High DIA GLD High SPY GLD High
Low DIA SPY Low DIA GLD Low SPY GLD Low
Close DIA SPY Close DIA GLD Close SPY GLD Close
Volume DIA SPY Volume DIA GLD Volume SPY GLD Volume

Conclusion

As expected there seems to be a relation between the DIA and SPY prices. Volumes differ by orders of magnitudes, which makes it hard to see any pattern in the charts. I can not say much about GLD. Next time I will perform simple statistical analysis to quantify any possible correlation. Oh, yeah, Grails is great and plugins too.

  • Share/Bookmark

July 18, 2010

Grails Finance 0.2

Filed under: programming — Tags: , , — admin @ 6:04 pm

Last week I made a scaffolded Grails finance application. This week I will fill the database with historical price data from Google Finance.

Model Changes

I needed to change the model a bit. I changed the name of the DateTime field and added some instance variables with corresponding constraints. Here is the modified FieldValue domain class

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

package finance
 
import org.joda.time.*
import org.joda.time.contrib.hibernate.*
 
class FieldValue {
    String val
    DateTime added
    Instrument instrument
    Field field
 
    static constraints = {
      val(blank:false)
      added(blank:false)
      instrument(blank:false)
      field(blank:false)
    }
 
    static belongsTo = [Instrument, Field]
 
    String toString() {
      "${val} ${added}"
    }
}

The view

I created a Groovy Server Page (GSP) that displays in a table all the Google Finance instruments in the database and buttons for data retrieval. Prototype is used to perform AJAX calls to the controller. I hope you recognize the famous Spinner UI design pattern.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

<html>
   <head>
      <title>Retrieve EOD data from Google Finance</title>
      <g:javascript library="prototype"/>
   </head>
   <body>
      <table>
         <g:each var="instrument" in="${instruments}">
         <tr>
            <td>${instrument.symbol}</td>
            <td>${instrument.name}</td>
            <td><g:form action='retrieve'>
            <g:hiddenField name='symbol' value='${instrument.symbol}'/>
            <g:submitToRemote value="Get data"
            url="[controller: 'fieldValue', action: 'retrieve']"
            onLoading="showSpinner(true)" 
            onComplete="showSpinner(false)"/>
            <img id="spinner" style="display: none" src="<g:createLinkTo dir='/images' file='spinner.gif'/>"/>
            </g:form></td>
         </tr>
         </g:each>
      </table>
      <g:javascript>
            function showSpinner(visible) {
                $('spinner').style.display = visible ? "inline" : "none"
            }   
        </g:javascript>
 
   </body>
</html>

The page looks like this in Safari

Instruments

The controller

The controller does all the hard work of querying for instruments and retrieving data in CSV format. The data consists of a Date, Open, HighLow, Close and Volume column. I installed the REST plugin for the HTTP communication part

1

grails install-plugin rest

The plugin injects amongst others the withAsyncHttp method.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52

package finance
 
import static groovyx.net.http.ContentType.HTML
import org.joda.time.*
import org.joda.time.format.*
import org.joda.time.contrib.hibernate.*
 
class FieldValueController {
    def scaffold = true
 
    def eodgoogle = {
      def instruments = Instrument.findAll().findAll { 
         instrument -> instrument.source.name == 'Google Finance'
      }
      [instruments : instruments]
    }
 
   def retrieve = {
      withAsyncHttp(poolSize : 4, uri : "http://finance.google.com", contentType : HTML) { 
         def result = get(path:'/finance/historical', query: [q:params.symbol, output:'csv']) { 
            resp, html ->  println ' got async response!' 
            return html
         } 
 
         assert result instanceof java.util.concurrent.Future
 
         while (! result.done) { 
            Thread.sleep(2000) 
         }
 
         def html = result.get().toString()
 
         for(line in html.split('\n')) {
            if(!line.contains('Date')) {
               def fields = line.split(',')
               def dateTime = DateTimeFormat.forPattern("dd-MMM-yy").parseDateTime(fields[0])
               def instrument = Instrument.findBySymbol(params.symbol)
               saveValueByFieldName('Open', dateTime, instrument, fields[1])
               saveValueByFieldName('High', dateTime, instrument, fields[2])
               saveValueByFieldName('Low', dateTime, instrument, fields[3])
               saveValueByFieldName('Close', dateTime, instrument, fields[4])
               saveValueByFieldName('Volume', dateTime, instrument, fields[5])
            }
         }
      }
   }
 
   def saveValueByFieldName(fieldName, dateTime, instrument, val) {
         def mnemonic = Field.findByName(fieldName)
         def fVal = new FieldValue(added: dateTime, val: val, instrument: instrument, field:mnemonic).save();
   }
}

A screenshot of the historical values list is shown here below. Mmm, gold …

Values

Notice that no effort is required to get pagination.

Conclusion

Grails is really perfect for secret home projects, especially for weekend ones. Next on the Grails finance roadmap is charting, some mathematical analysis, a REST webservice, lots of polishing and even more features, which will remain secret for now.

  • Share/Bookmark
Older Posts »

Powered by WordPress