An Adaptive Stock Tracker forPersonalized Trading Recommendations

Jungsoon YooComputer Science Department

Middle Tennessee State UniversityMurfreesboro, TN 37132 USA

+1 615 898 5737csyoojp@mtsu.edu

Melinda Gervasio and Pat LangleyInstitute for the Study of Learning and Expertise

2164 Staunton Court, Palo Alto, CA 94306+1 650 494 3884

{gervasio,langley}@isle.org

ABSTRACTThe Adaptive Stock Tracker is a personalized recommen-dation system for trading stocks that utilizes automatically acquired content-based models of user preferences to tai-lor its buy, sell, and hold recommendations. The system incorporates an efficient algorithm that exploits the fixed structure of user models and relies on unobtrusive data-gathering techniques. In this paper, we describe our ap-proach to personalized recommendation for stock tracking and its implementation in the Adaptive Stock Tracker. We also discuss experiments that evaluate the behavior of the system on both human subjects and synthetic users. The results suggest that the Stock Tracker can successfully adapt its recommendations to different types of users.

KeywordsAdaptive user interfaces, machine learning, user model-ing, personalization, information filtering

INTRODUCTIONWith the advent of the Internet, a wealth of information awaits anyone within the touch of a few keystrokes. Un-fortunately, the desired content is often buried in massive amounts of irrelevant information and it is left to each user to cull through the extraneous material. For example, through online brokerage firms, one can now check stock prices and make transactions through a Web browser. With all the stocks available throughout the world, there are more opportunities to make or lose money, but only if one has the time to follow all those stocks. Tracking tens of thousands of stocks is beyond the capability of any sin-gle user. Information filtering systems address the problem of infor-mation overload by filtering out irrelevant information and reducing the amount of information that the user must ex-amine. Such systems could give an investor more oppor-tunities to examine potentially profitable stocks by elimi-

nating obviously non-profitable stocks. This task of sepa-rating interesting from uninteresting information can be viewed as a classification task. However, since different people have individual tastes, information filtering should be personalized. This can be achieved through user mod-els or profiles that embody the preferences of a user or group of users. Moreover, such profiles can be learned from interactions with individual users. In this paper, we describe the Stock Tracker, an adaptive user interface that recommends stocks based on an indi-vidual’s trading profile. The Stock Tracker utilizes this profile to ranks stocks, and it revises the profile based on traces of user behavior. We have evaluated this prototype experimentally on historical records from the S&P 500 stock market and obtained positive preliminary results.In the next section, we define the stock tracking problem formally. We then discuss our technical approach, in-cluding the overall system architecture, engineering deci-sions, and implementation issues. Next we present some hypotheses about the Stock Tracker's ability to adapt to in-dividual users and describe the experiments we ran to test them.In closing, we review related work and consider some directions for future research.

THE PROBLEM OF ONLINE STOCK TRACKINGThrough the Internet, ordinary people can now have ac-cess nearly 3500 companies that make up the New York Stock Exchange, almost 4500 securities (stocks) in the NASDAQ composite, and many more available through various Web sites. The values of stocks change continu-ously, while news and trading information regarding a company’s financial status are available instantly. This makes it virtually impossible for anyone to keep a close watch on more than a handful of stocks.Many commercial sites partially address this problem by presenting buy recommendations on a small set of stocks. However, few of them make complementary sell recom-mendations, as this would have to be tailored to each user’s portfolio. Furthermore, different traders have dif-ferent investment styles, in that some are aggressive risk takers and others are more concerned with long-term re-

<LEAVE BLANK OF 1” high HERE>

turns. Existing sites do not typically cater to such individ-ual preferences.Our goal is to build an adaptive stock tracking system that acquires an individualized profile automatically through interaction with a user and utilizes this profile to make personalized trading recommendations. Adaptive stock tracking can be viewed as a specialized case of the more generic task of filtering events in a high-dimensional space. Given many continuous variables that change over time and online data for these variables sampled at regular intervals, the information filtering task is to find trends or events in the data that the user will find significant.What is “significant” depends on the specifics of the prob-lem domain. If the task involved monitoring events in a complex assembly plant, for example, a significant event may be a divergence from planned developments that re-veals the possibility of an accident. Significance also de-pends on the user and suggests a role for personalization. For example, depending on a user’s expertise, an informa-tion filtering system might alert the user at different times or present information at different levels. By automati-cally adapting its behavior to individual needs and prefer-ences, an information filter can further increase its utility as an assistant in complex temporal domains. We have built a prototype stock tracking system that learns personal preferences on top of a decision frame-work based on a pure technical analysis. That is, the anal-ysis uses only temporal stock trading data (e.g., stock price, trade volume), and not any financial information about the company [1]. Thus, we can formally state the stock tracking problem as a temporal information filtering task:Given: A user and his associated profile;Given: A large set of stocks available to buy and sell;Given: Online trading information for these stocks col-

lected at regular intervals; andGiven: Strategies for how to make a trading decision;Find: A list of trading recommendations in an order that

reflects the user's priorities.To make personalized recommendations, the Stock Tracker relies on user profiles that embody individual preferences. These profiles are acquired automatically by calling machine learning techniques on user interaction traces. The system gathers data unobtrusively, taking ad-vantage of an interface design that obtains useful informa-tion from a user’s natural interactions. In the next section, we present the details of our approach to adaptive stock tracking.

A PERSONALIZED APPROACH TO STOCK TRACKINGTo reiterate, the Stock Tracker addresses the problem of information overload through a personalized approach to information filtering. By considering personal user prefer-ences, the system filters trading information in an individ-ualized manner, presenting only information that a user finds interesting. In this section, we present the details of

the Stock Tracker. After presenting the architecture, we elaborate on the graphical interface, the recommendation module, and the user modeling component.

System Architecture The Stock Tracker is built on a client-server architecture, with information filtering, record keeping, and adaptation performed on the server, while the user interface and re-lated computing are done in the client, as shown in Error:Reference source not found1. The server contains the data processing unit, recommendation module, user modeler, information manager, and communication unit. The data processing unit converts raw input (i.e., current stock readings and historical trading information) into reports that contain buy and sell recommendations for the user. It relies on the recommendation module to make appropriate suggestions for each stock based on individual user pro-files. The user modeler constructs these profiles, utilizing user responses to previous recommendations. The infor-mation manager records traces of a user’s interactions with the system and also keeps track of user portfolios. Fi-nally, the communication unit manages the information into and out of the server.A client contains a communication unit and a graphical user interface (GUI) component. The communication unit performs activities that correspond to the server’s commu-nication unit. Meanwhile, the GUI presents all reports to the user and accepts commands such as buying/selling stocks and viewing portfolios, along with requests for ad-ditional financial information on particular companies. The system simulates trading using historical S&P 500 market data; it mimics a real stock trading scenario by generating information one (simulated) day at a time and letting a trader decide the stocks, if any, to buy and/or sell on each day.

An Interface for Unobtrusive Data CollectionThe Stock Tracker includes a graphical interface, shown in Figure 2, for presenting stock information, making rec-ommendations, and accepting the user's trading requests. The system’s ranked list of recommendations appears in the upper left. Details about the highlighted are presented in interactive graph on the bottom half of the window. The upper right presents a summary of the current stock,

Figure 1. Architecture of the Stock Tracker.

together with the system’s recommendation and action buttons for the user to buy or sell the stock. The user may also, but need not, provide additional feedback on the rec-ommendation—for example, clicking on “Thank you” to indicate agreement with a warning or specifying a desired alternative in the case of an unacceptable recommenda-tion.Although we have not systematically tested this user inter-face for usability, we have tried to adhere to known princi-ples, such as Shneiderman’s eight golden rules of interface design [18] and Karat’s principles of usability [7]). Since the system adapts its recommendations based on interac-tions with the user, we were also particularly concerned about its methods for obtaining user feedback. Some ap-proaches to personalization require users to state their preferences explicitly through long questionnaires or indi-cate to directly whether each recommendation is good or bad. Instead, we prefer unobtrusive data collection tech-niques and have designed the Stock Tracker's interface so that it can obtain useful feedback through a user’s natural interaction. For example, a user can provide positive feed-back by purchasing a stock that the system recommends he should buy. By selling the same stock, the trader gives negative feedback. Because more explicit feedback is also helpful, we also provide this facility, but the Stock Tracker can adapt its behavior to users even without such information.

Action RecommendationAt the heart of the Stock Tracker are the personalized trading recommendations that it makes to each user. The system bases these recommendations on a technical analy-sis called Moving Average Convergence Divergence (MACD) [1] that examines the difference between long-term and short-term moving averages to identify crossing points. These points indicate market turns and thus corre-spond to opportunities for buying or selling stock. Specif-ically, the time to buy stock is when both the long-term and short-term averages are increasing and the short-term average has exceeded the long-term average. Conversely, the time to sell is when the averages are decreasing and the short-term average has dipped lower than the long-term average. We compute short-term averages over nine days and long terms over eighteen. We use MACD be-cause it is a simple algorithm but still a popular technique in stock market analysis.We convert MACD into decision rules for recommending five different actions: buy, buy warning, sell, sell warning, and do nothing. The buy and sell rules correspond directly to those just described. A buy warning occurs when a rec-ommendation to buy is likely in the near future—that is, both averages are increasing but the short-term average re-mains lower than the long-term average. Similar logic ap-plies to the sell warning, while all other situations corre-spond to do nothing. Although MACD specifies the con-ditions for each action, the Stock Tracker must still deter-mine exactly when to make each recommendation with re-

spect to before/after the short-term average crosses the long-term average. Different choices correspond to differ-ent risk levels and capture different preferences about how soon to give buy and sell warnings. The MACD technique gives, in some sense, objectively optimal buy and sell rec-ommendations, but our primary objective in building an information filtering assistant is to help users make their own decisions more efficiently, rather than to make deci-sions for them.Each decision rule consists of a set of numeric constraints on temporal stock-trading attributes, such as the rate of in-crease in the long-term moving average or the difference between the long-term and short-term averages. A deci-sion rule applies when all of its constraints are satisfied—that is, the value of each corresponding attribute satisfies the constraint. For example, we can encode “the long-term average is increasing” with the constraint (LTA_slope ≥ ). By using a parameter instead of zero, we allow for differences in how much a stock price must be increasing for an individual to consider it. Thus, while MACD defines the form of the constraints, the Stock Tracker can alter its recommendation behavior by using different threshold values.

Personalizing Stock RecommendationThe Stock Tracker achieves personalized recommendation through the use of individual user profiles that capture stock trading preferences. A profile consists of four bi-nary classifiers, one for each action other than do nothing. A binary classifier renders a membership decision on each item (i.e., is this a positive instance of the class?). In the Stock Tracker, each binary classifier embodies a MACD decision rule and makes a recommendation if its con-straints are satisfied. For each stock, the recommendation module selects the recommendation with the highest asso-

Figure 2. Graphical user interface of the Stock Tracker.

Learn (examples)For each classifier in {buy, buy warning, sell, sell warning}:

LearnOne (classifier, examples, classifier’s constraints)

LearnOne (classifier, examples, constraints) Sort examples in increasing order according the

attribute value of the next constraint in constraints. Identify threshold candidates for splitting the

examples into positive and negative regions. Set constraint threshold of classifier to best split

among candidates according to Evaluate (examples, split). LearnOne (classifier, Subset (examples,split), remaining constraints)

Table 1. The Stock Tracker's learning algorithm.

ciated confidence value, or do nothing if no classifier rec-ommends an action. This confidence value is then used to present the selected recommendations on different stocks in a ranked list.The system builds classifiers from training examples ex-tracted from traces of the user’s interactions. Briefly, the user can either accept or reject each recommendation. An acceptance indicates that the recommendation was correct and is thus a positive example of the corresponding classi-fier. Similarly, a rejection produces a negative example. Each classifier can also use positive examples for other classifiers as negative examples for itself. Although there exist several well-known classification algorithms, we opted to devise a new, more efficient algorithm that ex-ploits the fixed structure of the user model, as Figure 3 summarizes.Recall that each decision rule (classifier) consists of a set of constraints, each of which corresponds to a particular numeric attribute. Each constraint specifies a threshold on the attribute value, above (below) which the constraint is satisfied. The goal of the learning algorithm is thus to find a set of thresholds that will result in recommendations consistent with the user’s actions. If we order the exam-ples according to increasing attribute values, we can eval-uate candidate thresholds for a ≥ constraint based on how well they predict positive examples above the threshold and negative examples below it, and similarly for ≤ con-straints. Also, since the constraints form a conjunctive set of conditions, thresholds for the remaining constraints need only be considered for examples in the region that satisfies the constraint.

We tried a variety of methods for evaluating candidate splits, including information gain and various metrics based on precision and recall, which are often used to evaluate information retrieval systems.We found the F measure [12], a weighted combination of precision and re-call, to provide the best behavior. Precision indicates the probability that a positive instance labeled as positive by the classifier is truly positive, whereas recall gives the probability of correctly identifying all positive instances. In our case, precision is the number of positive examples in the correct partition divided by the number of instances in that partition, while recall is the number of positive in-stances in the correct partition divided by the number of positives in both partitions. A primary motivation for the development of an efficient algorithm is that learning is conducted online—that is, af-ter every interaction with the user that yields positive or negative examples, the user modeler updates the user pro-file. This lets the Stock Tracker adapt quickly to individ-ual traders. To give the system a reasonable starting point, we employ a default model, corresponding to the profile of an average user, that it induces from a default data set that represents feedback from such a user. By at-taching a weight to these default data, we can vary the de-gree to which the system relies on them during its model construction.

EXPERIMENTAL EVALUATIONOur goal in developing the Stock Tracker was to help users more quickly identify stocks to buy or sell, based on their individual interests. By using its interaction with the user to adapt its model of the user’s trading preferences, the Stock Tracker learns to tailor its recommendations to the user. With experience, the Stock Tracker should thus

Figure 3. Performance of the Stock Tracker with human subjects in terms of acceptance rate (left) and time spent on each transaction, with curves indicating trend/regression lines.

become better at recommending actions a user will accept. Here, we describe the experiments we conducted with hu-man and synthetic subjects to test this primary hypothesis.

Experiment with Human SubjectsThe basic question we want to answer is: Can the Stock Tracker learn to tailor its recommendations to different users? We can evaluate the performance of the Stock Tracker by measuring its success in recommending trading actions on different stocks for different individuals. Specifically, for each individual, we can count how many times an individual accepts the system’s recommenda-tions. More accepted recommendations translate to better performance. We can also measure how long it takes a user to complete a transaction. If the Stock Tracker makes good recommendations and ranks them highly, users should require less time to complete transactions. Thus, our hypothesis was that over time, for any individ-ual user, the acceptance rate would increase while the time per transaction would decrease.To test this hypothesis, we conducted an experiment with twelve subjects, with a varied range of stock trading ex-pertise. The users attended a brief orientation session to learn how to use the system, and were asked to run a prac-tice session to familiarize themselves with the system. The experiment simulated daily trading using 2001 S&P data, with subjects given the option of completing the ex-periment over multiple sessions.1 Each subject began the experiment with $20,000 in their portfolio, roughly half of which was already invested in selected stocks. As a user interacted with the Stock Tracker, we measured the time taken to complete each transaction and the acceptance rate of recommendations, where the acceptance rate is the number of user actions matching the recommendations over the total number of user actions.Figure 3 shows the change in acceptance rate as the num-ber of user actions (i.e., number of training cases) in-creases. At first glance, the results are somewhat surpris-ing—they show the acceptance rate going down before climbing back up. They are also a little disappointing, with the eventual acceptance rate not going much higher than the initial acceptance rate. However, through an analysis of user traces and interviews with the subjects, we determined that the initial acceptance rate was artificially inflated by the somewhat unexpected behavior of new users. At the start of using the Stock Tracker, new users tended to focus on building their portfolio, basically ignor-ing the initial portfolio we had provided. They did this mostly by selecting from the system’s buy recommenda-tions, which resulted in the high initial acceptance rate. But as they started to monitor these stocks, users also started making transactions in disagreement with the sys-tem’s recommendations, leading to the dip in acceptance rate. As the Stock Tracker gained experience with the

1 We asked each user to use the system until at least 50 transactions/suggestions are generated.

user, however, it started making better recommendations, thereby increasing the acceptance rates again.Figure 4b shows how the average amount of time a user spent on each transaction decreased with the number of user actions. This provides further evidence of successful adaptation. Even though users were urged to spend some time familiarizing themselves with the Stock Tracker be-fore running the experiment, we suspect that the decreased transaction time is partly due to users becoming more fa-miliar with the system anyway. However, we contend that in conjunction with the previous result on acceptance rate, this is also due to the system learning to rank acceptable recommendations more highly and thus reducing user ef-fort in finding good suggestions.

Experiment with Synthetic SubjectsWhile the previous experimental results are promising, in-terviews with the subjects revealed a potential problem with novice users. These users expressed that they some-times had trouble figuring out what to do in the first place. Thus, it is quite likely that the adaptation results were con-founded by novice users simultaneously learning about in-vestment themselves. The Stock Tracker is designed for knowledgeable traders, who have distinct preferences about the stocks to invest in. Thus, to better evaluate the system’s ability to adapt to different users, we designed an experiment to evaluate its performance on such a popula-tion of users.The basic question we want to answer is: Can the Stock Tracker learn to tailor its recommendations to experienced users? Because of the difficulty in finding knowledgeable test subjects, we opted to use synthetic users for this ex-periment. Recall that the Stock Tracker learns user mod-els of a fixed form based on the MACD technical analysis. By varying the threshold parameters in the model, we can effectively create users with different investment styles. The actions predicted by these synthetic models could be interpreted as the actions made by the corresponding syn-thetic users. As in the previous experiment, we use the dependent variable of acceptance rate to measure success.2

Our hypothesis was that acceptance rate would increase over time.To test this hypothesis, we ran an experiment on 200 syn-thetic investors, ranging from very conservative to very aggressive. We tried to match the previous experimental setup by testing each user in a simulated online fashion on 2001 S&P data. Each user started at a randomly chosen date from the first fifty days of the year with the same ini-tial portfolio. Each user model was initialized with a de-fault data set. For each day, the Stock Tracker used the current user model to generate a list of recommended transactions, ordered by their associated confidence value as discussed earlier. The list was filtered to exclude any sell/sell warning recommendations for stocks the user did

2 Since we were using synthetic users, transaction time is not meaningful.

not own. To simulate regular user behavior, the system then randomly picked for evaluation five of the top ten recommendations plus one other from the rest of the list. The recommendations of the current user model on these stocks were then compared to the actions actually taken by the user (i.e., predicted by the synthetic user model). As before, the number of matches over the total number of actions taken measured the system’s acceptance rate. Fi-nally, these stocks and the user’s actions were used to up-date the user model in preparation for the next day. For each user, we also ran a control condition where no adap-tation occurred (i.e., the default model was not modified by user interaction).Error: Reference source not found shows the Stock Tracker’s acceptance rate over time, averaged over the 200 users. The results show that the system achieves an acceptance rate of about 98% after less than thirty user ac-tions (i.e., training examples). In contrast, without learn-ing, the Stock Tracker has an acceptance rate of only about 88%.3 These results support our hypothesis that the Stock tracker can successfully adapt to experienced in-vestors with different trading preferences.

Experiment on Default ModelAs mentioned in the previous experiment, the Stock Tracker initializes new user models with a default data set of training instances. This is primarily to provide reason-able recommendations even for a new user. However, the

3 There is a slight upward trend in the acceptance rate, in-dicating that, in general, the actions for the stock trading situations in the latter part of the year were easier to pre-dict for the default model. This could be because the latter part of the year was more similar to the 2000 S&P data on which the default model was based, or it may simply be the nature of the stock market.

use of a default model can affect the Stock Tracker’s abil-ity to adapt to users. A crucial issue with adaptive sys-tems, particularly those that learn online, is rapid adapta-tion. Thus, it is important that a default model achieve good initial performance without sacrificing its ability to quickly learn individual preferences.

Figure 4. Acceptance rate of Stock Tracker recommendations, with and without learning.

In using a default model, there are three questions we need to answer. The first is: What makes a good default user (model)? Preliminary experiments showed that a moder-ate user (i.e., neither too conservative nor too aggressive) offered the best combination of making good recommen-dations without making too conservative recommenda-tions overall. The second question is: How large should the default data set be? A large data set will provide good performance for users similar to the default but not for very different ones in addition to slow response time. We settled on a default data set size of 200 after preliminary experiments showed that it offers reasonable starting per-formance for different types of users.The third question, and the one we set out to answer here is: What is an appropriate weight setting on the default data set? A smaller weight gives more importance to the data generated by user interaction—for example, a default weight of 0.1 means it takes ten default examples to have the same effect as one example from a user action. Given different weight settings, we thus want to measure the ac-ceptance rate—in particular, the initial acceptance, the as-ymptotic acceptance, and the rate of convergence. Our hypothesis was that an intermediate weight on the default model would make the best tradeoff between initial accep-tance rate and rapid adaptation.To test this hypothesis, we created a default data set using 200 randomly chosen instances from the 2000 S&P data, which we labeled with the predictions of a moderate syn-thetic user. We then ran an experiment on 200 synthetic users, in a simulated online fashion on 2001 S&P data as in the previous experiment. We tested three different de-fault model weights: 1, 0.1, and 0.01. As hypothesized,

the best balance between initial and asymptotic behavior came with the intermediate weight (0.1) (Error: Referencesource not found). This had nearly as high an initial ac-ceptance rate as the high weight (1) while also achieving adaptation almost as fast as the low weight (0.01). Thus, we used this weight setting for the default model in the previous experiment on synthetic and human users.

RELATED AND FUTURE WORKMuch of the research on adaptive information filtering has focused on document retrieval or text categorization, espe-cially for Internet-related recommendation applications—for example, Web pages [2,13], e-mail [16], and news [4,9]. Unlike the problems tackled by these approaches, stock tracking is a temporally sensitive task, requiring the continuous monitoring of (numeric) variables and detect-ing trends or changes over time. This requires identifying features that capture these changes and employing meth-ods to gather such information. Addressing somewhat dif-ferent issues of changes over time are Klinkenberg & Renz [8], who investigate the issue of concept drift (i.e., changing user preferences) and Languillon [11], who stud-ies the problem of information drift (i.e., changing do-mains).Machine learning approaches to adaptation for recommen-dation are often divided into collaborative approaches and content-based approaches. Collaborative approaches [17] make recommendations based on user similarity, as evi-denced by the similarity of their item ratings, while con-tent-based approaches [10] make recommendations based on item similarity, as evidenced by their content descrip-tions. Collaborative approaches perform well when there is significant overlap between the users and the items they

Figure 5. Acceptance rate for the Stock Tracker using different default weights for the user model.

have rated. They perform less well for users with idiosyn-cratic behavior and cannot make recommendations on new items for which none or few ratings exist. Content-based approaches like the Stock Tracker do not suffer from these limitations as they learn individualized preferences over item attributes. However, they require content descrip-tions of the items and are thus better suited to domains where such information is readily available (see [10] for examples). Research on combining collaborative and con-tent-based techniques (e.g., [2,3]) attempts to address their individual limitations and could also be useful in our work.Earlier systems for personalized recommendation often imposed a significant burden on users to provide explicit feedback regarding the desirability of the recommenda-tions. More recent work has proposed solutions ranging from asking the user to rate the most informative items [14], using synthetic users (filterbots) to augment the user pool [15], and using implicit feedback to approximate ex-plicit ratings [5,6,13]. Rather than use low-level events such as mouse and keyboard tracking to indicate interest or approval, the Stock Tracker relies on the implicit feed-back that comes naturally in the course of the user inter-acting with the system—that is, accepting recommenda-tions to buy/sell or executing contradicting actions.The experimental results presented in the previous section provide evidence of the Stock Tracker’s ability to rapidly adapt to users with different investment styles. However, there remain additional issues to investigate. We need to replicate the study on experienced synthetic users to hu-man subjects, focusing on knowledgeable investors. An-other open question is whether the MACD-based structure of the user model is sufficient to capture a wide range of user preferences. Experiments with human subjects should help answer this question. The profiles used in the user modeling component of the Stock Tracker can be eas-ily replaced with any one of many technical analysis methods available [1]. We can design synthetic users for the chosen model type and evaluate the Stock Tracker’s ability to adapt to them. An additional open question is how to capture complex models. Models that are more complex could be captured by imposing a hierarchy of the stock market such as the hierarchy defined in the Global Industry Classification Standard (GICS, [6]). The Stock Tracker could be ex-tended to adapt to each user’s preference on each node in the hierarchy; this will capture different preferences of a user on different types of stocks such as a software com-pany stock vs. an automobile company stock.In summary, the Adaptive Stock Tracker is an information filtering system that learns to rapidly tailor its trading rec-ommendations to users with different investment styles. It does this by automatically acquiring a model of user pref-erences through an efficient algorithm that exploits the fixed structure of the user model. The Stock Tracker does not require users to fill in lengthy questionnaires about

their preferences or provide detailed feedback about its recommendations. Instead, it relies on unobtrusive data collection techniques to extract training data from its natu-ral interaction with the user. Experimental results with human and synthetic subjects provide support for the ap-proach, showing that the Stock Tracker can successfully learn to make increasingly acceptable recommendations as it interacts with individual users.

REFERENCES1. Achelis, Steven B. Technical Analysis From A To Z.

Probus Publishing, Chicago, 1995.2. Balabanovic, M. and Shoham, Y. Fab: content-based

collaborative recommendation. Communications of the ACM (1997), 40:3, 88-89.

3. Basu, C., Hirsh, H., and Cohen, W. Recommendation as classification: using social and content-based infor-mation in recommendation, Proc. of AAAI ‘98 (Madi-son WI), 714-720.

4. Billsus, D. and Pazzani, M. A personal news agent that talks, learns and explains. Proc. of Agents ‘99. (Seattle WA).

5. Claypool, M., Le, P., Wased, M., and Brown, D. Im-plicit interest indicators. Proc. of IUI 2001 (Santa Fe NM), 33-40

6. The Global Industry Classification Standard available at http://www.spglobal.com/GICSIndexDocumen-t.PDF.

7. Goecks, J., and Shavlik, J. Learning users’ interests by unobtrusively observing their normal behavior. Proc. of IUI 2000 (New Orleans LA), 129-132.

8. Karat, J. Evolving the scope of user-centered design. Communications of the ACM (1997), 40, 7, 33-38.

9. Klinkenberg, R. and Renz, I. Adaptive information fil-tering: learning in the presence of concept drifts. Proc. of the AAAI ‘98/ICML ‘98 Workshop on Text Catego-rization.

10.Konstan, J., Miller, B., Maltz, D., Herlocker, J., Gor-don, L., and Riedl, J.. GroupLens: applying collabora-tive filtering to usenet news. Communications of the ACM (1997), 40, 3, 77-87.

11.Langley, P. User modeling in adaptive interfaces. Proc. of UM ‘99 (Banff Alberta), 357-370.

12.Lanquillon, C. Information filtering in changing do-mains. Proc. of the Workshop on Machine Learning for Information Filtering (Stockholm Sweden, 1999), 41-48.

13. Lewis, D.D., Schapire, R.E., Callan, J.P., and Papka, R. Training algorithms for linear text classifiers. Proc. of SIGIR ‘96, 298-306.

14.Lieberman, H. Letizia: an agent that assists Web browsing. Proc. of IJCAI ‘95 (Montreal Canada), 924-929.

15.Rashid, A.M., Albert, I., Cosley, D., Lam, S., McNee, S.M., Konstan, J.A., and Riedl, J. Getting to know you: learning new user preferences in recommender sys-tems. Proc. of IUI 2002 (San Francisco CA), 127-134.

16.Sarwar, B.M., Konstan, J.A., Borchers, A., Herlocker, J., Miller, B., and Riedl, J. Using filtering agents to im-prove prediction quality in the GroupLens research col-laborative filtering system. Computer Supported Coop-erative Work, 1998.

17.Segal, R., and Kephart, J. MailCat: An intelligent as-sistant for organizing e-mail. Proc. of Agents ‘99 (Seat-tle WA), 276-282.

18.Shardanand, U. and Maes, P. Social information filter-ing: algorithms for automating 'word of mouth'. Pro-ceedings of CHI ‘95 (Denver CO).

19.Shneiderman, B. Designing the user interface. Strate-gies for Effective Human-Computer Interaction, Third edition. Addison-Wesley, Reading MA, 1998.