Watermarking Large Language Model-based Time Series Forecasting

View on arXiv ← Back to list

Authors: Wei Yuan, Chaoqun Yang, Yu Xing, Tong Chen, Nguyen Quoc Viet Hung, Hongzhi Yin

Published: 2025-07-28 12:16:52+00:00

AI Summary

This paper proposes Waltz, a novel post-hoc watermarking framework for Large Language Model-based Time Series Forecasting (LLMTS) to address intellectual property protection and misuse concerns. Waltz embeds imperceptible watermarks by modifying the similarity between time series patch embeddings and 'cold' LLM tokens, achieving high detection accuracy with minimal impact on forecast quality.

Abstract

Large Language Model-based Time Series Forecasting (LLMTS) has shown remarkable promise in handling complex and diverse temporal data, representing a significant step toward foundation models for time series analysis. However, this emerging paradigm introduces two critical challenges. First, the substantial commercial potential and resource-intensive development raise urgent concerns about intellectual property (IP) protection. Second, their powerful time series forecasting capabilities may be misused to produce misleading or fabricated deepfake time series data. To address these concerns, we explore watermarking the outputs of LLMTS models, that is, embedding imperceptible signals into the generated time series data that remain detectable by specialized algorithms. We propose a novel post-hoc watermarking framework, Waltz, which is broadly compatible with existing LLMTS models. Waltz is inspired by the empirical observation that time series patch embeddings are rarely aligned with a specific set of LLM tokens, which we term ``cold tokens''. Leveraging this insight, Waltz embeds watermarks by rewiring the similarity statistics between patch embeddings and cold token embeddings, and detects watermarks using similarity z-scores. To minimize potential side effects, we introduce a similarity-based embedding position identification strategy and employ projected gradient descent to constrain the watermark noise within a defined boundary. Extensive experiments using two popular LLMTS models across seven benchmark datasets demonstrate that Waltz achieves high watermark detection accuracy with minimal impact on the quality of the generated time series.

Key findings

Waltz achieves high watermark detection accuracy (high F1 scores) across seven benchmark datasets and two LLMTS models with minimal impact on forecasting performance (low MSE and MAE). The watermark remains detectable even after model distillation, demonstrating robustness against unauthorized use.

Approach

Waltz uses a post-hoc watermarking approach. It identifies 'cold' tokens in the LLM that rarely align with time series data and embeds watermarks by subtly altering the similarity between time series patch embeddings and these cold tokens. Projected gradient descent is used to minimize the impact on forecast accuracy.

Datasets

ETTh1, ETTh2, ETTm1, ETTm2, Electricity, Exchange, Weather

Model(s)

TEMPO, UniTime, DLinear (used for distillation attack)

Author countries

Australia, China

← Previous