\documentclass[Afour,sageh,times]{sagej}

\usepackage{moreverb,url}
\usepackage{graphicx}%
\usepackage{multirow}%
\usepackage{amsmath,amssymb,amsfonts}%
\usepackage{amsthm}%
\usepackage{mathrsfs}%
\usepackage[title]{appendix}%
\usepackage{xcolor}%
\usepackage{textcomp}%
\usepackage{manyfoot}%
\usepackage{booktabs}%
\usepackage{algorithm}%
\usepackage{algorithmicx}%
\usepackage{algpseudocode}%
\usepackage{listings}%
\usepackage{subcaption}

\usepackage[colorlinks,bookmarksopen,bookmarksnumbered,citecolor=red,urlcolor=red]{hyperref}

\newcommand\BibTeX{{\rmfamily B\kern-.05em \textsc{i\kern-.025em b}\kern-.08em
T\kern-.1667em\lower.7ex\hbox{E}\kern-.125emX}}

\def\volumeyear{2026}

\begin{document}

\runninghead{Yom-Tov}

\title{Classifying Sparse Time Series with Rich Payloads via Temporal Alignment}

\author{Elad Yom-Tov\affilnum{1}}

\affiliation{\affilnum{1}Bar Ilan University, Israel\\

\corrauth{Elad Yom-Tov,
Department of Data Science and Artificial Intelligence,
Bar Ilan University,
Ramat Gan 5290002,
Israel}

\email{elad.yom-tov@biu.ac.il}

\begin{abstract}
Time series with rich payloads (TSRP) are sequences of irregularly sampled, high-dimensional observations in which temporal alignment is informative but sample sizes are often limited. Such data arise in numerous domains, yet existing classification methods are ill-suited for these challenges. This setting falls outside the assumptions of most existing time-series classifiers, which rely on dense numerical observations or large training samples.

We present a framework for classifying TSRP that integrates representation learning with sparse Dynamic Time Warping (DTW) and support vector machines (SVMs). High-dimensional payloads are first embedded into lower-dimensional representations, after which sparse DTW aligns sequences for SVM-based classification. 

Our approach is evaluated on six datasets derived from social media, capturing medically relevant transitions such as shifts in mental health status. Results demonstrate that the proposed framework substantially outperforms a baseline based of average pooling across five of six datasets. We further analyze performance relative to dataset characteristics, provide insights into model behavior, and introduce an interpretable mechanism linking temporal alignments to input features. 

Overall, this work establishes a general, interpretable, and effective approach for classifying sparse time series with rich payloads, providing a foundation for applications in healthcare and other high-stakes domains.
\end{abstract}

\keywords{time series classification, pattern recognition, sparse DTW, embeddings}

\maketitle

\section{Introduction}
Time series with rich payloads (TSRP) are data consisting of time-stamped entries, wherein each data point is accompanied by a payload of diverse measurements and/or unstructured content. Such data are prevalent in a variety of domains, including health, commerce, education, and transportation. Examples include social media posts written by an individual, a user's search engine queries (where the payload is, e.g., the text of queries), and a patient’s comprehensive medical history, as recorded in their Electronic Health Records (EHRs). Leveraging these data streams holds the potential to forecast important outcomes, including health conditions \citep{soldaini2017,white2017,youngmann2019}, product purchase intent \citep{esmeli2021}, and the propensity for customer attrition \citep{huang2012}. 

While this wealth of information presents an opportunity for extracting insights and predict outcomes, existing approaches for analyzing such data often rely on ad-hoc methodologies, failing to fully harness the richness encapsulated within these payloads and the temporal aspects of the data. Moreover, analysis is further complicated because, in many cases, TSRP data are sparse and non-uniformly sampled, are multimodal (consisting, for example, of both text and images), and limited in their abundance owing to the difficulty in collected these data. 

Here we focus on TSRPs with textual payload. Current approaches to the analysis of such TSRPs focus on one aspect of the data (temporal or payload) and largely ignore the other aspect of the data. In this, two large bodies of work can be applied to analyze TSRPs: Time-series prediction and text analysis for prediction. Time-series prediction methods assume that the data are collections of (multidimensional) measurements of real-valued signals, commonly assumed to be sampled at equal time intervals \citep{fu2011review}. 

Several algorithms have been proposed for classifying time-series data (or more generally, sequence data \citep{xing2010brief}) into an outcome \citep{keogh2024,mori2017reliable,yeh2017matrix}, e.g., one of several discrete classes. Recent work has begun examining cases where data is low-dimensional irregularly (non-uniformly) sampled data \citep{li2020learning,shukla2021heteroscedastic}, for example, structured data in EHRs \citep{gupta2021concurrent,hu2020new}. 

TSRPs are differentiated from time-series data in several ways. First, the payload means that each data point is very high-dimensional and (often) sparse. Second, the data points are rarely generated at a fixed sampling rate. For example, search engine queries are generated whenever a user has an information need \citep{carmel2006makes} that they assume can be answered by a search engine. Finally, the data cannot be assumed to be measurement signals. While in some cases the data comprise of measurements such as blood pressure or heart rate, in other cases it is not, for example, when the payload is textual or a mix of textual and measurement data. 

Text mining methods are usually applied to discrete units such as documents, paragraphs, or sentences in order to elicit insights from the text \citep{feldman2007text}, for example, to predict an outcome. However, the temporal aspect is often lost in text analysis. Thus, for TSRPs, where the temporal aspect is one of their defining features, text mining methods are lacking. For example, marketing researchers refer to the process of purchasing a new product as that of a conversion funnel \citep{ramos2008search}, where people experience a staged process in order to purchase a product or service. Analysis of searches people made prior to purchasing \citep{rothschild2025} will miss a significant dimension of the data if it does not take the temporal aspects into account.

One exception, where time is explicitly taken into account in text mining, is in the online generation of summaries for time-critical news events \citep{guo2013updating}. Here, individual text items are received over time and a single summary is generated from these items. The goal of an algorithm is to generate a salient summary of information about a given event at each time instance, by selecting which text to extract from news items which arrive over time. However, the goal of these algorithms is to create a useful summary of information for a human reader, not to predict the outcome given the information.

Research on the combination of time-series analysis and textual analysis is insufficiently studied and applications of outcome prediction to time series with unstructured (especially textual) payload should be made more principled. One of the most common ways in which such data are analyzed is using bucketing of time-delimited windows. Specifically, all the text inside a given range of time is taken together and analyzed using methods of text analysis (e.g., \citep{de2016discovering,hochberg2019can,white2017}). Another common method is to project the payload into a low-dimensional space and aggregate the data in that space. For example, Youngmann et al. \citep{youngmann2019} trained a neural network to summarize a payload, in that case mouse movements during a search engine session, which were then used to predict whether or not a user is likely to have Parkinson’s Disease. In other work, Yom-Tov et al. \citep{yom2024identifying} developed an algorithm for screening for amyotrophic lateral sclerosis (ALS) by training a model to predict the likelihood that a user has ALS from data of a single session (the payload). Individual sessions were then aggregated using minimum pooling. A similar approach was reported for evaluating disease onset in the context of mental health \citep{viani2021natural}.

It is important to acknowledge another limitation in the analysis of TSRPs in real-world applications, which is that, in many cases, the data which can be used to train the model are small, meaning that only hundreds \citep{yom2024identifying,youngmann2019} to thousands \citep{white2017} of samples are available for training. This precludes the naive use of modern methods of deep neural networks (e.g., \citep{Yang2023}), as there is insufficient data to train such algorithms. 

Thus, our goal in this work is to develop the performance of a method for prediction based on time-series data with diverse payloads, which can be applied even when only limited training data is available. Our focus in this work is on analysis of textual internet data, especially for medical purposes. Though we are by no means limited to these data and this application, we do this for their central importance to the understanding of human behavior and their practical applicability to the benefit of people’s health. First, we note that internet data are ubiquitous and span the gamut of human behaviors \citep{yom2016crowdsourced}, including, for example, domestic violence \citep{zaman2021discovering,youngmann2023intimate}, loneliness \citep{mazuz2020analyzing}, pregnancy \citep{fourney2015exploring}, mental health \citep{rissola2021} and various economic indicators \citep{choi2012predicting}. Second, internet data have been used to study health because they offer a window into this topic in an unprecedented manner, due to their extensive use by diverse populations \citep{yom2016crowdsourced}. As such, they have found application in affecting public health \citep{sadilek2018machine,vaidyanathan2022evaluation,weintraub2023identifying} and individual health. 

Individual health applications can often be posed as time-series prediction with payload. A case in point is screening for disease from internet data (i.e., search engine queries, public images, or social media posts), where the goal is to predict the likelihood that a user is afflicted with a medical condition before they have been diagnosed by a medical professional. This application has garnered significant attention and was shown useful for a variety of conditions including anxiety \citep{youngmann2018anxiety}, depression \citep{schueller2020understanding} and post-partum depression \citep{de2014characterizing}, suicidality \citep{cheng_search_2019,de2016discovering}, eating disorders \citep{sadehsharvit_predicting_2020}, sleep disorders \citep{cohen_zion_identifying_2022} and stroke \citep{shaklai_detecting_2021}. 


\section{Predicting from sparse time-series data with payload}

Our approach in this work is to transform the TSRP prediction problem into a time-series prediction problem, by casting the payload data into a low-dimensional signal. Once thus transformed, we measure similarity between different time series using previously-proposed methods for sparse time-series, and use these as input to a distance-based classifier.

\subsection{Notation}

Let $D_{i,t}$ be a sample from time-series $i$ at time $t$. Time-series $i$ may represent the samples from the entity of interest, for example, user $i$. Each sample $D$ in the time-series has a payload such as text, image, or video. Each time-series is a member of a class $C$, for example, healthy or ill. Denote by $y_i$ the class of time-series $i$. Here we assume that $C \in \{0, 1\}$.

The goal of our algorithm is to predict which class a time-series belongs to, given $D_{i,t}$. 

\subsection{Embedding payload data into a lower dimension}
\label{sec:embedding}

The first stage of the process is to embed $D_{i,t}$ into a lower dimensional signal $x_{i,t}$. There are multiple ways in which this could be achieved, ranging from a simple indicator of the existence of a sample (ignoring the actual payload) to multidimensional embeddings such as BERT \citep{gardazi2025bert} or similar. 

Here we propose to use the probability that the payload was made by a member of the positive class. That is, given the payload, $x_{i, t} = P(y_i = 1 | D_{i, t})$. This one-dimensional embedding extracts information from the payload (if such information exists), while creating a simple representation for the time-series. 

\subsection{Calculating the distance between pairs of time-series}
\label{sec:distance}

Having embedded the payload into a low-dimensional signal, we can now calculate the distance between pairs of time-series. That is, given $x_{i, t}$, calculate the pairwise distance matrix $M_{i, j}$. The nature of our data is that the time-series are highly sparse and this needs to be taken into account when calculating these distances. 

We chose to use the sparse version of the well-known Dynamic Time Warping (DTW) algorithm. DTW is a measure of similarity between (possibly) misaligned time sequences. DTW for sparse sequences can be estimated \citep{mueen2016awarp} by run-length encoding of the missing data and calculating DTW on the resulting sequences. 

Since the output of DTW is a distance, not a similarity, the distances are transformed into similarities using a transform such as a negative exponent. 

It is noted that Mueen el at. \citep{mueen2016awarp} assigned the same weight to the missing data and to the existing samples in the data. Here we give the existing data a weight factor, $w$, relative to the missing data. This can be done by multiplying the existing data by $w$ before calculating DTW.

Once the distance between all pairs of time-series is calculated, a distance matrix $M=M_{i,j}$ is generated which can be used to classify the time series into their most likely classes.

\subsection{Classifying time-series}
\label{sec:classification}

Many classifiers use pairwise distance matrices as input. These include nearest neighbors, Gaussian Processes and Support Vector Machines (SVMs). Here we focus on the use of SVMs, which have been shown to provide good classification accuracy on a large number of datasets \citep{fernandez2014we}. 

SVMs have other attractive properties in that they are easily extended to include multiple kernel matrices via Multiple Kernel Learning (MKL) \citep{gonen2011multiple}. These are applicable to our case, as they allow incorporating multiple embeddings, distance measures, or weights ($w$) into the classification process, possibly improving the accuracy of the classifiers. 

For example, if multiple embeddings are used in the first part of the process (Section \ref{sec:embedding} above), the pairwise distance can be calculated for each embedding dimension, and the resulting pairwise similarity matrices can be combined using MKL. Alternatively, if multiple weights are assigned to the existing data, compared to missing data (Section  \ref{sec:distance}), these can be combined at this stage using the same MKL approach.

Therefore, given $M_{i,j} \forall i, j$ and $y_i \forall i \in Training Data$, the result of this section is a classification of each test sample into the most likelihood class. 

\section{Interpreting the predictions}

Interpretability \citep{gilpin2018} is an important feature of a classification process, not least because it allows domain experts to review the classification models and comment on how they operate vis-a-vis their domain knowledge, and, perhaps more importantly, to shed new light on the domain of interest. 

Interpretability is challenging when the data is sparse and the payloads are rich. Thus, here we offer a method to provide interpretable results from the classifiers trained in the previous sections. 

Given a distance matrix $M$ (or a similarity matrix), the samples are clustered using a pairwise similarity algorithm such as k-medians \citep{ackermann2010clustering}. 

Once the data are clustered, for each cluster we select a representative sample (a medoid) such that its average distance to all other samples in its cluster is minimal. We further require that the sample has at least $n$ samples in its time series ($n$ is a parameter set by the user).

To visualize the medoids they are shown on a temporal axis. The existing data points are smoothed using a Gaussian kernel to generate a dense time series. In this way, the explanations show how different user prototypes change in their similarity to the target class over time. 

\section{Methods}

\subsection{Data}

We extracted data for this study from Reddit, a social network. On Reddit, people post to communities with shared interest, known as subreddits. 

Our data includes postings made on Reddit between 2015 and 2024. The data consists of six datasets, each following a specific structure: A user first posts (one or more times) to a designated source subreddit, and in some cases, subsequently posts to a different subreddit, referred to as the target subreddit. The initiation of posting to the target subreddit is considered clinically significant. For example, following De Choudhury et al. \citep{de2016discovering}, we extracted all postings on Reddit's depression subreddit (r/depression), which was the source subreddit of this dataset. Our label was whether an author began posting on the suicidal ideation subreddit (r/SuicideWatch), which served as the target subreddit in this case. 

Samples from the source subreddit are taken up to the first time that a post is made in the target subreddit. The times of the posts were normalized to time relative to the first target post (e.g., one day before the first target post is $t=-1$). For authors who only posted in the source subreddit and never in the target subreddit, a random reference time was chosen. 

One exception to the above is the Trying to Conceive (TTC) dataset, where the target subreddit were indications of a successful pregnancy made in the same source subreddit on specific weekly threads (the "Weekly success thread"). 

We created 6 datasets using this methodology, as described in Table \ref{tab:data}.

\begin{sidewaystable}
    \caption{Datasets analyzed. The time series are created from the postings in the start subreddit. Authors are classified as positive if they started posting in the target subreddit. The percentage of sparsity is the number of days on which there were samples over the entire data period. The percent positive is the percentage of authors who started posting in the target subreddit. }
    \label{tab:data}
    \centering
    \begin{tabular}{cllccc}
        \hline
        \hline
        Index & Start & Target & Number of & Percent & Percent \\
              & subreddit & subreddit & authors & sparsity & positive \\
        \hline
        \hline
1   & COVID19 & Long COVID19 & 290 & 2.1 & 87.2 \\
    & (r/COVID19positive) & (r/LongCovid, r/covidlonghaulers) & & & \\
\hline
2   & Diet and weight loss & Weight loss surgery & 918 & 2.9 & 6.0 \\
    & (r/OzempicForWeightLoss, r/WegovyWeightLoss, & (r/BariatricSurgery, r/gastricsleeve) & & & \\
    & r/loseit, r/diet, r/omaddiet, r/keto, & & & & \\
    & r/keto\_diet, r/intermittentfasting) & & & & \\
\hline
3   & Intimate partner violence (IPV) & Divorce & 618 & 2.2 & 17.5 \\
    & (r/domesticviolence, r/emotionalabuse, & (r/divorce, r/Divorce\_Men,  & & & \\
    & (r/abusiverelationships, r/NarcissisticAbuse) &  & & & \\
\hline
4   & Trying to conceive (TTC) & Pregnancy & 2639 & 4.9 & 42.4 \\
    & (r/TTC30) & (r/TTC30 success posts) & & & \\
\hline
5   & Pregnancy & Premature birth & 3953 & 3.9 & 9.0 \\
    & (r/pregnant, r/BabyBumps, r/TryingForABaby, & (NICUParents) & & & \\
    &  r/amipregnant, r/PregnancyAfterLoss, r/IVF, & & & & \\
    &  r/BabyBumpsCanada, r/pregnancyproblems) & & & & \\
\hline
6   & Depression & Suicide watch & 1666 & 4.0 & 56.8 \\
    & (r/Depression) & (r/SuicideWatch) & & & \\
        \hline
        \hline
    \end{tabular}

\end{sidewaystable}

The work described herein was approved by the author's Institutional Review Board.

\subsection{Analysis}

The embedding of Section \ref{sec:embedding} was made using a Random Forest with 500 trees. We refer to this model as the {\em base classifier}. 

Two variants of the SVM model of Section \ref{sec:classification} were tested: One with a single distance matrix and the other with two distance matrices with different $w$ values: 1 and 100. 

We compare the proposed modeling framework to a baseline which uses average voting (the average of the probabilities computed in Section \ref{sec:embedding}) as the predicted class membership.

Classifier performance was evaluated via the area under the Receiver Operating Curve (AUC). Ten-fold cross-validation was used to reduce the chance of an overfit \citep{duda2000pattern}. 

We tested the performance of the algorithm as a function of the relative weight of existing data ($w$) and the SVM cost parameter ($C$) by conducting a parameter sweep on values from $10^{-3}$ to $10^{3}$. 

Additionally there are several parameters of the dataset whose correlation with the AUC of the model was tested. The parameters are:
\begin{enumerate}
    \item {\bf Smoothing filter duration}: The probabilities computed in Section \ref{sec:embedding} are averaged across several days. We tested the following number of aggregation days: 1, 3, 5, and 15. 
    \item {\bf Minimum number of non-zero samples}: Authors are kept only if they have at least $k$ samples, where $k$ are 1, 3, or 5. 
    \item {\bf Dataset size}: The number of time series in the data
    \item {\bf Sparsity}: The average percentage of non-zero entries in the time series.
\end{enumerate}

The results of the parameter sweep for the first two parameters (and their effect on the latter two) was modeled using rank regression to elicit the association of the individual parameters with the model's AUC. 

\section{Results}

Figure \ref{fig:param_sweep} shows the effect of the SVM cost parameter $C$ and of the relative weight of non-zero samples compared to missing data ($w$) on AUC. As the figure shows, except for one dataset (TTC to pregnancy), lower values of $C$ result in higher AUC, while higher values of $w$ result in higher AUC. Given these results, we used $C=0.01$ and $w=100$ in subsequent results.

\begin{figure}[tb]
    \begin{subfigure}{\linewidth}
    \centering
    \includegraphics[width=0.6\linewidth]{Figs/param_sweep_C.jpg}
    \label{subfig:a}
    \end{subfigure}
    \begin{subfigure}{\linewidth}
    \centering
    \includegraphics[width=0.6\linewidth]{Figs/param_sweep_W.jpg}
    \label{subfig:b}
    \end{subfigure}
    \caption{The effect of the SVM cost parameter ($C$) (top) and the relative weight of non-zero samples compared to missing data ($w$) (bottom) on AUC. The horizontal axis is $C$, $w$, respectively, and the vertical axis in each graph is the AUC. In all graphs the smoothing filter duration is 5 and the minimum number of non-zero samples is 5. In the top graph $w=100$ and in the bottom graph $C=0.01$. }\label{fig:param_sweep}
\end{figure}


Figure \ref{fig:results} compares the AUC of the proposed algorithm in its two variants to the baseline algorithm. As the figure shows, the proposed algorithm is better than or equal to the baseline in all but one dataset. 

\begin{figure}[t]
  \centering
  \includegraphics[width=0.75\linewidth]{Figs/auc_for_datasets.jpg}
  \caption{AUC for different datasets and algorithms. The blue bars show the AUC for the average pooling model. The red bars for the proposed model with a single distance matrix, and the yellow bar for MKL approach.}
  \label{fig:results}
\end{figure}

Table \ref{tab:paramModel} shows the model parameters for a rank regression model where the independent variables are the parameters of the datasets and the dependent is the AUC of the proposed model and (separately) the difference between the AUC of the proposed model and the AUC of an average pooling model. As the table shows, the proposed model works better when the data are small (has fewer samples) and sparser. Additionally, a better baseline classifier results in superior classification accuracy. However, the difference between models also becomes smaller as baseline classifiers improve. 

\begin{table}[tb]
    \centering
    \begin{tabular}{lcc}
        \hline
        Parameter & AUC of  & AUC of model  \\
        & model & minus AUC of  \\
        & & average pooling \\
        \hline
        Number of samples & -0.57 & -0.64 \\
        AUC of base classifier & 0.54 & -0.17 \\
        Smoothing filter length & -2.33 & -1.32 \\
        Minimum number of samples & -4.99 & -5.92 \\
        Sparsity & 0.60 & 0.33 \\
        \hline
    \end{tabular}
    \caption{Rank regression model parameters. All results are statistically significant at $P<0.05$ with Bonferroni correction.}
    \label{tab:paramModel}
\end{table}

The latter effect is further investigated in Figure \ref{fig:worse_classifier}. Here, the performance of the base classifier was modified by applying random forest models with fewer trees (between 1 and 500) to two datasets: COVID to long COVID and Depression to Suicide Watch. As the figures demonstrate, better baseline classifiers result in higher AUC for both the proposed model and the average pooling model, but the gap between them is also reduced as the models improve. 

\begin{figure}[tb]
    \begin{subfigure}{\linewidth}
    \centering
    \includegraphics[width=0.6\linewidth]{Figs/classifier_performance_vs_base_classifier.jpg}
    \label{subfig:worse_classifier_a}
    \end{subfigure}
    \begin{subfigure}{\linewidth}
    \centering
    \includegraphics[width=0.6\linewidth]{Figs/classifier_performance_diff_vs_base_classifier.jpg}
    \label{subfig:worse_classifier_b}
    \end{subfigure}
    \caption{Dependence of the proposed classifier AUC (top) and the difference between the AUC of the proposed classifier and an average pooling classifier (bottom) on the base classifier AUC for two datasets: COVID to long COVID and depression to Suicide Watch. Also shown are linear regression lines. In the top graph the $R^2$ is 0.87 and 0.24, respectively. In the bottom graph the $R^2$ is equal to 0.78 and 0.34, respectively. }\label{fig:worse_classifier}
\end{figure}

As noted above, interpretability is an attractive feature of any classification model. Specifically, we refer to the ability to provide information about the model to domain experts such that they can review the model and perhaps obtain new information about the problem. To achieve this, we proceeedeed as described above, with $N=5$ clusters and a minimum of 20 points in a time series and a Gaussian width of 10 days. 

Figure \ref{fig:clustering} shows examples of the explanations for two of the datasets, depression and IPV. For each cluster we show the number of users in it and the likelihood of a cluster member to be in the positive class. Thus, referring to the depression clusters, it can be seen that people who post frequently with texts that make them likely to be in the positive class are likely to transition to that class (clusters 5 and 2). However, some users exhibit similar behavior (cluster 1) but only around two thirds of them transition to the positive class. Alternatively, members in clusters 3 and 4 post for long periods in the depression subreddit, but their posts do not resemble those of the target class and indeed, very few of them transition to this class. 

Referring to the IPV dataset, we observe that people tend to post for long periods of time in IPV subreddits without transitioning to divorce (clusters 2, 4 and 5), compared to others who post with higher likelihood of transitioning to divorce (clusters 1 and 3) with only a minority of users choosing to do so. 

Thus, the proposed clustering can offer a view into the different modes of user behavior in the different datasets. 

\begin{figure}[tb]
    \begin{subfigure}{\linewidth}
    \centering
    \includegraphics[width=0.5\linewidth]{Figs/suicideWatch_clusters.jpg}
    \label{subfig:clustering_a}
    \end{subfigure}
    \begin{subfigure}{\linewidth}
    \centering
    \includegraphics[width=0.5\linewidth]{Figs/IPV_clusters.jpg}
    \label{subfig:clustering_b}
    \end{subfigure}
    \caption{Clustering medoids for the depression (top) and IPV (bottom) datasets. Black dots represent data points. Each color represents one of the clusters. The caption shows the number of users per cluster and the likelihood of a cluster member to be in the positive class. }\label{fig:clustering}
\end{figure}


\section{Discussion}

Time-series data enriched with payload information (TSRP) are prevalent in a variety of domains, including health, commerce, education, and transportation. However, very few principled methods exist for their analysis, especially for predicting an outcome from these data.

Here we focused on a specific domain of TSRP, namely, medicine. Our datasets span a range of medical applications, ranging from pregnancy to depression. In all cases, there is an important medical prediction question, related to a future behavior such as the transition from posting in the depression forum to starting to post in the suicidal ideation forum.

The proposed method consists of three stages: First, the payload is transformed into a low-dimensional signal. Then, the similarity between (sparse) time series is estimated using DTW. Finally, this similarity is used as a kernel for an SVM classifier. We further propose a method to use the similarity (or distance) matrix and create a visual explanation of the dataset, through k-median clustering and a smoothing in the time-domain of the selected medoids.

Our results demonstrate that the proposed method is superior to average pooling, especially when the data are small and sparse. Interestingly, though the proposed model offers better performance than average pooling, the gap in performance becomes smaller when the base classifier is more accurate. This can be explained if we assume that a perfect base classifier would perfectly predict the class of a user given a single sample. In that case, there is no need to capture the temporal aspect of the data, and average pooling would perform similar to the proposed method. However, the gap in performance is not completely closed because the temporal aspects do matter, and the base classifier is far from perfect for all datasets, meaning that it is not possible to perfectly predict the outcome solely using the payload. 

We note that the use of multiple kernels (MKL) provides only a slight improvement over a single kernel. Future work will investigate whether including other kernels (for example, based on different modalities of the payload or different base models) can provide more significant advantages over a single kernel.

One of our datasets was similar to that of De Choudhury et al. \citep{de2016discovering}. In that paper they used single posts from the depression subreddit to predict the transition to suicidal ideation. In that work, the reported AUC was 0.87. While the datasets are not directly comparable (due to different timeframes and different preprocessing and analysis methods), we note that average pooling in our work achieved somewhat lower AUC (0.72) and the proposed method reached an almost perfect AUC of 0.97. Thus, the integration of temporal aspects seems to improve classification accuracy significantly. 

Clustering of the data reveals interesting insights into the data. For example, in the movement from the depression subreddit to the suicidal ideation subreddit, it is notable that people can cease posting information that indicates their likelihood to transition to the suicidal ideation subreddit, but do so with high likelihood around 2 months later. This points to the importance of continuing to monitor people whose mental state seemingly improves, but may, unfortunately, transition to suicidal ideation some time after that improvement. This is in agreement with past work \citep{roy2020machine}, which found that peaks in risk marker tweets tend to occur 1–10 days before a suicidal ideation events. Additionally, our results reveal that engagement in a depression subreddit by itself is not indicative of future transition to suicidal ideation. 

Future work will examine the benefit of multimodal data for predicting of outcomes. Multimodal data is easily incorporated with the proposed model, since it only requires additional base models. We are also investigating how the proposed method can be applied to dense and noisy data such as search engine query data, to predict outcomes such as gynecological cancer \citep{barcroft2024using}. In those data, each payload contains less information and most payloads are irrelevant. Thus, more work is required for the proposed method to be applied to them.

Nevertheless, the proposed method represents a significant advancement in prediction from TSRPs and provides insights into the underlying data. As such, it is useful for both researchers and practitioners in a range of domains.


\bibliographystyle{SageH}
\bibliography{bibliography}

\end{document}
