Beyond the Timestamp--How to Spot Financial Predictions

Paper or topic, venue, and link (state of the art only)

This paper Automatic detection of relevant information, predictions and forecasts in financial news through topic modelling with Latent Dirichlet Allocation was published in the Applied Intelligence Journal in 2023. It focuses on the intersection of natural language processing, knowledge extraction, and financial news analysis, utilizing Latent Dirichlet Allocation (LDA), temporality analysis, and personal finance management.

The problem and why it matters

Financial related news for investors is an ongoing collection of data from various sources. The collection and extraction processes are tedious due to investors manually monitoring and extracting the relevant material. Manually extracting matters for investors as a delay in financial market updates can result in a loss of investment, which can lead to many more problems. In addition, what they extract can be unstructured as data is from various sources.

Many financial NLP systems instead lean on timestamps (i.e., “as of May 10”) and treat temporality only at the surface level through verb tenses (i.e., “announced,” “is trading,” or “will rise”). In contrast, this paper matters because it performs temporal analysis over the discourse itself—meaning it looks at how verbs and their relationships are used in the surrounding sentence to express predictions/forecasts tied to an asset.

The method in your own words

Since previous works do not focus on the temporal analysis of considering verbs as semantic concepts, this work here utilizes the verbs as features or feature inputs to their ML model, namely the support vector machine (after testing on many of the models). Before using the verbs as feature inputs, the authors first majority vote verb tenses (past, present, future) with auto selecting future if tie exists. To extract these verbs and their tenses, they perform dependency parsing along with their Algorithm 1, which points to which terms within the input sentence rely on each other by also looking at their part of speech. In particular they denote FutDepSubObj and FutProxSubObj as future-tense, thus a prediction. FutDepSubObj is defined as the number of future tense verbs from the dependency analysis when the asset is the subject or object of the clause. FutProxSubObj is defined as the number of future tense verbs from the proximity analysis when the asset is the subject or object of the clause.

What is genuinely novel

Algorithms 1 describe the generation of the temporal features of a segment based on the dependency.

Reproduction: what you ran, the setup, what worked or broke, results or plots

The code and datasets were not provided by the authors. I have reached out to request the dataset and am currently waiting for a response.

Limitations and the "why didn't they do X" gaps

One limitation is they don’t provide access to code and dataset, so we can’t test if results are reproducible or not. Another is not providing their distributions of past, present, future, so we can’t relabeled according to our own work nor test on our models. No direct classification on prediction or not as they used these future related sentences as input to the SVM or ML temporal analysis.

Ideas this sparks for our work

The paper's focus on temporal analysis directly informs the development of TOLSA-M (Target Outcome with optionaL Source, dAte, and Metadata), particularly in addressing the limitations of current predictive systems. While previous works often only capture the future tense to identify predictions, TOLSA-M expands this scope by including past and present tenses. This allows for the tracking of a narrative over time—such as monitoring how a specific prediction evolves from a season opener through the end of a season. Furthermore, the paper's approach to using verbs as semantic context to identify forecasts aligns with the challenge of identifying TOLSA-Ms, which requires navigating the relationship between predictive-like statements and various discourse genres, such as beliefs, claims, events, and facts. By integrating these temporal dimensions and discourse connections, TOLSA-M aims to move beyond simple keyword matching (e.g., searching for "prediction" or its synonyms) to a more granular extraction of measurable outcomes and their associated temporal properties.