Project Update_iVC Reporting Engine

/

Still working industriously behind the scenes I thought to take some time and give a progress report. Good news is the iVC Reporting Engine is almost fully operational. I've been able to automate the following processes:

Data Collection: I have two methods to obtain public company filings from the SEC via Python scripts.

  • The primary method I use leverages the excellent services of the free (for now) data provider, Quandl.com. They aggregate and distribute data from several primary sources including the SEC and normalize as best as possible given whatever business constraints they have as a young riser. In congruence with their mission statement to improve data accessibility for all, they have also provided a Python module and general API for the motivated entities out there to automate data queries. 
  • Additionally I have a backup database of every public company filing since 2012, which leverages Arelle's open-source efforts to standardize and improve the XBRL standard for finance-IT adoption. My db is configured to use the PostgreSQL standard with Arelle's schema. This still requires more examination of the XBRL format, and/or command line interfaces to return the data I require. Marked: in Development for production purposes.
  • Data Processing: There is overlap with the above, however this includes organizing/filing of the data returned by the collection process. Not a lot to say here other than the files are self-organizing and allow for flexible structures during continued operational development. 

Calculation Engine: This is pretty much the final major component before large scale testing can begin. It is broken down as follows. 

  • I've been able to transition ~99% of my previous Excel and Python calculations into this component. It accesses the processed data, and strips it down to run the calculations as efficiently as possible currently averaging ~15.5 secs per cycle. 

Under Construction: All that's left really is to incorporate the plotting/charting functions and final production output format. Once that is complete live production testing can begin.

You better tell somebody, Blackarbs LLC is coming. 

​

Walking the Path takes time...

/

Readers, 

I have been away somewhat after my declaration of providing more useful and interesting content. Why is that? Am I being hypocritical or something worse?

The answer is neither. The essence of the Blackarbs mission is to continuously find ways to add value to investors anywhere and everywhere they may be. Born out of that mission is the iVC report which has helped a few portfolio managers, and private investors (including myself) make prudent well timed investments in value creating stocks (Details to come in a future post). 

To better help current and prospective clients I relentlessly evaluate the current reports, the report creation process and its efficacy as a research tool. I've concluded that the stock reports are incredibly effective at identifying value creators and destroyers within the public equity space but the current, report creation process is NOT SCALABLE. 

Unfortunately I am unable to produce these reports on a massive scale because it takes to long to generate one. When I first began production it took ~4-5 hours to produce a single report. Now it can be done in ~1-1.5 hours which is a dramatic improvement but not nearly fast enough. If there are roughly 3000-5000 stocks in our potential investment universe we can see the time investment needed to evaluate even a subsection of stocks is too great.

THE SOLUTION:

It's simple, the report creation process needs to be automated as much as possible. As a result I've spent the last 12 days working on this programmatically having experimented with solutions based primarily in Python. After building 2 different web scrapers and a calculation engine I realized each 'free' fundamental data source I found presented unique challenges due to their html/javascript structure or some strange reporting convention they use.

Me in the lab 


Enter XBRL. If you are not familiar XBRL is the cutting edge financial reporting standard based on the XML documentation 'language' that the SEC has mandated public companies adhere to. Essentially publicly traded firms have a mandate to provide all their fundamental SEC/financial data in this semi consistent format which is ..."easily"... accessed  by software. 

Make no mistake as a result of being 'cutting edge' and in development, the XBRL standard is not simple or easy, yet. However, it is the best solution in that the calculation engine I use to analyze the companies and output grades will be based on data provided directly to and by the SEC! This is a massive upgrade IMHO to third party data providers. This project is ongoing and Blackarbs' progress in developing and implementing this solution is fast-paced. 

So please forgive the sporadic posts until the report generation engine is completed hopefully within the next two weeks.