What you can find here:
- A step by step best practise process to create a sound ranking system
When it comes to designing ranking systems, it is not possible to supply an exact recipe that will always assure success. But we can provide guidance that will help you recognize and constructively address all the important choices that need to be made. Besides doing that, we also supply some checklists and factor catalogs that can help you organize your efforts and avoid information overload.
For experienced investors, the concepts you’ll find here will not be earth-shattering. In fact, much will rehash things you already know about stock selection. The focus, here, is on organization, how to approach the task of ranking system design in such a way as to help you see how ideas you already have can be expressed in the language of portfolio123, to help you easily identify the factors and formulas likely to be of interest to you, and perhaps most important, how to steer clear of concepts that aren’t compatible with your investment philosophy and thereby avoid information overload.
The topic of ranking-system design is, essentially, the same as what makes a good stock. Such a discussion can go on and on, as you can often see in our Community and elsewhere. Consider, for example, the differences in approaches advocated by the likes of Jim Cramer and Warren Buffett. If you listen and/or read carefully, you'll see that you could create ranking systems consistent with either philosophy. Both appreciate fundamentally sound companies so in either case you'll want some factors based on this. Beyond that, a Cramer-oriented system would require sensitivity to what's working now, or as he puts it, what can and cannot be owned. Hence it would contain significant helpings of estimate-based, recommendation-based, and price-based factors. A Buffett-based system, on the other hand, would tilt toward consistency. Such a system would use lots of three-, five-, or even ten-year factors as opposed to quarterly or trailing 12 month items and have heavier weightings of fundamental factors, as well as more sensitivity to valuation.
As is the case with many endeavors, the best way to enhance your rank-design capabilities is to practice, practice, practice. The preceding paragraph is an example of how you can do it no matter where you are, even if you are not logged into portfolio123. Any time you encounter someone's investment philosophy, whether in conversation, while watching TV, while listening to the radio or a podcast, or while reading, consider, in your own mind, what sort of ranking system you might create to implement it. Don't worry abut getting it "right." There are usually many ways to implement a particular approach. The key is to hone your proficiency in translating investment philosophies into factors and formulas.
With that understanding of the nature of our task, let's now turn to specific steps that can help organize our implementation efforts.
Choose your theme(s)
Start by establishing a menu from which you will choose. Here's the one I use:
Companies that grow more rapidly than others.
This theme stresses the relationship between the stock price and one or more important fundamental metrics. Can include income investing.
This approach emphasizes factors and formulas that suggest we are dealing with a "good" company. That doesn't necessarily mean its stock is flying high, or even that its stock is reasonably priced at the moment. Nor does it mean that the company is performing well right now; even the best of firms have down periods. Quality-oriented investing can require more patience than usual.
How key investment-community constituencies feel about a stock and more importantly, how they are acting. In contrast to quality, this approach opens the door to bad companies whose stocks seem likely to move.
This menu of themes involves combinations of considering a company versus its stock, and considering the here-and-now versus a longer-term perspective.
Here and Now
Obviously, the foregoing is geared toward fundamental analysis. That’s because this is my background and the way I’m most comfortable working. If you’re interested in technical analysis, the substance of what you do will, obviously, differ. But I suggest you take note of the process, which transcends differences between fundamentals and technicals. For example, if you were interested in technical analysis, as you would have to be to create ETF ranking systems, your thematic menu might look something like this:
This approach emphasizes prices action that more or less follows an identifiable trend. Your underlying assumption is that there are substantive reasons for this trend (which you may or may not care to articulate) and that it is likely to continue. The functions you choose for your rank formulas will generally be those that measure the strength of the trend, its persistence (longevity), and its sustainability (i.e. you’ll likely establish criteria that are geared toward sending early warning signals of a potential change in trend).
This approach is based on the idea of mean reversion. It assumes that stocks move up and down but not far from some sort of central trend and that if it moves too far away, it’s likely to reverse course and head back toward center. Stocks that move too far below a central trend are said to be oversold and ripe for purchase. Stocks moving too far above the trend are said to be overbought and vulnerable to correction. What about seemingly overbought stocks that are in fact, likely to shift toward new higher trading ranges? That is analogous to fundamental investors debating the merits of value versus growth. Discussions like that can go on and on and sometimes generate hostility, and at times, and in the hands of a creative ranking-system designer, actually be reconciled in models that accommodate both viewpoints.
These are very near-term approaches that look for oddities like crossover, gaps or breakouts; events that suggest something interesting is afoot. Presumably, whatever it was that suddenly caught the market’s attention will lead to a new trend in the direction of the oddity; for example, a new and perhaps more pronounced uptrend following a gap up. This theme lends itself well to Boolean (true-or-false) rank factors. For example, you may want to use, as a rank factor: GapUp(50,500,20,0), which means there was a gap up of at least 10% with a 500% volume increase from the 20-bar average. Stocks meeting this criteria will be ranked True, which merits a rank score of 100. Stocks that don’t meet the criteria will be scored zero. In a sense, this true-false approach allows you to bring a screening flavor into the ranking process.
The material presented here will use the fundamental thematic menu presented above. But you need not stick with it if it doesn't appeal to you, as would obviously be the case if you use technical analysis. What’s important is that you work form some sort of menu that’s relevant to you (perhaps you might use a different way to classify fundamental themes). Once you have your thematic menu in place, a useful first step in designing a ranking system is to state what theme(s) you will use. A system can confine itself to a single theme. Or it can use as many as you wish (even all available themes). If you use multiple themes, I suggest creating a composite (folder) for each as well as sub-composites for sub-themes.
Whatever your investment philosophy, I recommend you proceed step by step with the following six decision points:
- Thematic Focus
- Performance Sensitivity
- Comparison Criteria
- Specific Factors/Formulas
Let’s examine each of these decision points in more depth, using fundamental investing for purposes of illustration.
1. Thematic Focus
This is where you'll decide whether you're going to be a specialist or a generalist.
The decision is being made with reference to the ranking system as a whole, and with reference to each theme you chose to include. The goal here is to decide whether you'll use a small or large number of factors. Specifically, the choices are:
This means being a specialist. Choose this if you want very specific types of companies, such as growth. This doesn't mean you'll spend your life investing in only one kind of company. You may have many specialized systems all seeking different things. What it does mean is that each individual system will be very particular about what it seeks. In practical terms concentrated ranking systems are those with small numbers of factors and/or formulas.
This is probably the most mainstream approach. Such systems will reflect multiple themes, such as value-growth, and/or more than one way of expressing a particular theme. You'll have more than a few factors and/or formulas, but not so many as would raise eyebrows among others to whom you show your system.
These are generalist eyebrow-raising systems containing very large numbers of factors and/or formulas spread across most or all of the themes in your menu. These are for investors who want companies they select to be good in some respects, but they don't care deeply about which respects those might be. It can be a controversial approach - statisticians frown on it since these systems are very likely to have many factors that correlate with one another, and, hence, can be deemed redundant. Practically speaking, it's easy when using such systems to wind up giving more weight to a particular characteristic than you might realize (you weight factor X by 1 percent, but have many other factors that correlate to it such that underlying characteristic in effect has a 10 percent weight in your system). Advocates of comprehensive systems could rest on the fact that data-driven analysis such as we, in effect, use when we build rankings systems, can be imprecise (since so many aberrant things can happen to individual companies). Accordingly, it can be beneficial to be flexible in terms of how an individual firm demonstrates the general merit we seek.
2. Performance Sensitivity
This is another controversial topic in the area of ranking system design. Here, you are choosing the criteria by which you'll measure the success or failure of your system based on the performance tests you run.
Everyone wants a system where highly ranked stocks outperform poorly ranked stocks, regardless of how many "buckets" are used in the tests. You'll also want to see your well-ranked buckets outperform the market benchmark you choose. Those are the easy issues, the ones on which we can all agree.
The hard part is identifying a time period. Here are the choices:
- Immediate environment
This means you'll want your performance tests to show that the system is performing well now, or in the very recent past. You understand that such a model may be geared toward an atypical investment climate and might not perform as well in many, or perhaps most, other periods. But you're willing to maintain an "adaptive" approach to your investments. In other words, you'll use different ranking systems at different times, based on "heuristics," (subjective judgments about what type of systems are most relevant to the present market environment) and/or other macro or timing models you create elsewhere. If you succeed at such an approach, you'll enjoy spectacular performance. The risk is that you get whipsawed; that you suffer poor performance as you always aim at the market environment from which we just emerged. Investors who follow this approach rarely use more than a one-year time frame for their testing, and many will use less.
- Over a moderate time period
This is similar to the adaptive approach described above, except that it's implemented in a more casual manner. Here, you take a broader view of the market climate and recognize that your time horizon will include some transitions into and out of the conditions you deem most relevant. Accordingly, you will wind up with more instances of lackluster performance in your test results. But when you switch to live implementation, your exposure to whipsaw is diminished (although not entirely eliminated). Performance testing, here, is usually done in time frames ranging from one to three years.
- Over a long time period
These are the users most likely to click on "Max" when inputting the time frame for their tests (or if not max, then perhaps five years). Such investors do not want to engage in any sort of timing (except, perhaps, for higher-level decisions as to how to allocate funds between stocks and other asset classes). When it comes to stocks, these people want systems that have demonstrated an ability to work more often than not over longer time periods. They are more tolerant of cold spells than those who use the other approaches, but they are more likely to enjoy consistency.
It doesn't matter which approach you choose. Interestingly, though, this decision can engender vigorous argument since most investors are passionate about whatever decisions they make in this regard (hence the antagonism between, for example, Cramer fans versus Buffett devotees who are at opposite ends of this spectrum).
What is important - downright critical in fact - is that you make a conscious decision one way or the other and remember it through your testing, and most importantly, as you invest real money based on it. The last thing in the world one who favors immediacy needs is to lose track of that determination and stay too long with a system designed for a very specific set of conditions. Conversely, one who chooses a longer-term approach needs to remember this in tough times, in order to avoid self-induced whipsaw (moving out of a perfectly good system at the wrong time in response to a temporary cold period).
3. Comparison Criteria
If you experimented with the portfolio123 rank interface, you've already seen the choices:
Here, the company’s rank for a particular factor or formula will depend on how it compares to all other companies in the universe as a whole.
In this case, a company rank will depend on how it compares in the relevant factor or formula only to other companies in the same industry. In this case, Company A, with a growth rate of five percent, may rank higher than Company B and its 20-percent growth rate. That could happen if Company A is in a slow-growing, or perhaps declining industry in which many companies have growth rates worse than five percent. Company B, on the other hand, may be in a hot industry wherein most peers grow faster than 20 percent per year.
This is similar to the industry approach except that here, companies are being ranked relative to others in the same sector. When using industry comparisons, semiconductor firms would be ranked only in relation to other semiconductor firms. Here, with sector comparison, semiconductor firms would be ranked relative to all those in the technology sector, which includes not just semiconductors but other areas such as computer hardware, communications equipment, electronic instruments and controls, and so forth.
As with other decisions we face, you can make one choice or mix and match as you see fit. What's most important is that you recognize the implications of the choices.
Industry, and to a lesser extent, sector-based approaches are most associated with longer-term approaches to investing, since you may find great companies that happen to be in cold industries getting the highest ranks. From an investment point of view, this can be an excellent thing. But it may require more patience.
This isn't inevitable. Such ranking systems can also produce better companies in hot industries. If you want this, and are willing to accept all that goes with having a more adaptive immediacy-oriented system, consider adding some industry factors into your model. In other words, don't content yourself to rank companies based on trailing 12 month EPS growth relative to their respective industries. Add a factor based specifically on industry average TTM EPS growth, where industries will be raked relative to other industries.
If you rank companies relative to the entire investment universe, you will also bring more of an immediacy flavor to your models, since companies in hot industries will be more likely to rank well against the entire universe than will companies in cold industries.
With either of these immediacy leanings, stay sensitive to industry diversification. It's very easy to wind up with portfolios that are too heavily exposed to hot industries (another prescription for whipsaw). We can easily, and automatically, guard against this when we create simulations and portfolios. If you invest with real money before learning how to use those aspect of portfolio123, remember to stay alert on your own to the possibility of inadequate sector/industry diversification.
Now that you've made key decisions that will determine the general structure of your ranking system, it's time to get into the nity grity. This is where you will decide how to express the thematic decisions you made.
Suppose, for example, you are a growth investor. What, exactly, does growth mean to you? Do you want to see demonstrated track records of growth over three- or five-year time horizons? Or are you looking for growth in a trailing 12 month or perhaps one-quarter time frame? Maybe you want all of the above. All are valid expressions of the growth philosophy. This may seem like a difficult choice. Actually, though, assuming you gave proper attention to decisions regarding thematic focus and performance sensitivity, you should find it easy to gravitate to the sub-themes you want.
The hard part is keeping track of what sub-themes are possible. As with the main themes, each investor can come up with his or her own list. Below is the one I use.
- Performance (based on earnings, cash flows, etc.)
- Assets (based on cash, book value, etc.)
- Aggressive (may overlap other value sub-themes)
- Conservative (may overlap other value sub-themes)
- Dividend Current
- Dividend Growth
- Dividend Safety
- Balance Sheet
- Short Interest
You may wish to set up sub-composites for each of the sub-themes you use.
Now it's time to get specific. This is where you choose the specific factors and/or formulas you will use in your ranking system.
To the uninitiated, this can seem daunting. But if you set up folders and sub-folders for your themes and sub-themes, you should find the factor/formula selection process to be very comfortable. Rather than wading through the seemingly countless possibilities available to you on portfolio123, you will know what areas you want to go to.
To help you navigate the many choices available to you in portfolio123, catalogs of factors/formulas for each of the above mentioned themes and subthemes are offered in the Appendix. Feel free to copy any you want to use into your clipboard and paste directly into the ranking interface.
Weighting is the most judgmental (some might say trial and error) aspect of the process. There are statistical techniques available to calculate the best possible ("optimal") set of weights. I am not necessarily a fan of those approaches (which are not part of portfolio123). They raise a considerable risk of "data mining;" over-analysis of a specific time period to produce results that may not be relevant in other time periods. Producing "robust" models, ones that will work across many different market environments, remains a difficult task requiring, perhaps, as much or more art versus science.
Bearing in mind that the contents of each folder and sub-folder must have weights that total 100%, I suggest starting with equal weights across the board. Then, go through your items one at a time to determine how confident you are that better scores really are related to better stock price performance. Instances of more confidence can translate to higher weights and vice versa.
Mindful of the views of the quant/MPT crowd, I suggest that ultimately, everything you do here is based on confidence. If you have complete confidence in a particular factor, then you should use it alone with a 100 percent weight. (Similarly, if you have complete confidence in a stock, then you should invest 100 percent of your asserts in it.) We shiver thinking about doing such things for one reason and one reason only: because we do not have complete confidence. Notwithstanding all the fancy theory and higher mathematics, that is the essence of allocation/weighting/diversification decisions: adapting to the fact that our confidence levels do not warrant 100 percent commitments.
The dichotomy, here, is whether you measure your level of confidence through statistical studies of the past, or through your own judgment. Depending on who is in the crowd, those can be fighting words (as is the case with performance sensitivity choices, the issues here can be highly controversial). In my opinion, based on having worked with the statistical approaches, you aren't really missing anything when you don't optimize.
When I did work this way, I found myself constantly having to chase my own tail as all the seemingly good work I did collapsed as soon as I changed sample periods, and I’m not alone: some academics have been trying to enhance the Nobel-Prize-winning Capital Asset Pricing Model by working on something known as the C-CAPM (Conditional Capital Asset Pricing Model). Essentially, they’re chasing their tails the way I did, after having seen how betas and risk premiums bounce around all over the place as different periods are examined. There’s a lot of elitism attached to super-duper platforms that let you optimize but I really found this to have been a frustrating, miserable, and ultimately unproductive way to work. Do not feel all deprived by having to base factor selection and weightings on common sense judgment. In fact, you have the upper hand.