LingBot-Touch: A Foundation Model for Heterogeneous Tactile Representation Learning
LingBot-Touch is a JEPA-based tactile representation model that bridges heterogeneous tactile signals and dexterous robotic manipulation. It uses modality-specific tokenizers and a shared Transformer to learn representations of contact structure and temporal changes through prediction in latent space.
The model covers three sensing modalities: magnetic, piezoresistive, and visuotactile.
- Magnetic: three-axis tactile measurements.
- Piezoresistive: scalar tactile measurements placed in the Z channel, with X and Y set to zero.
- Visuotactile: decoded scalar deformation or intensity measurements placed in the Z channel, with X and Y set to zero. Raw RGB images are not model inputs.
Model Bundles
| Model | Encoder Weights | Inference Configuration |
|---|---|---|
| LingBot-Touch-Small-clip16 | encoder.pt | config.json |
| LingBot-Touch-Small-clip32 | encoder.pt | config.json |
Each bundle contains a matched config.json and encoder.pt. The configuration describes the inference architecture and modality routes; the weight file contains only the frozen target encoder's parameter tensors. Keep the two files together.
This model repository distributes weights and their necessary inference configurations only. It does not include inference source code, sensor adapters, sample datasets, predictors, downstream task heads, optimizer states, training configurations or training logs. The compatible LingBot-Touch inference implementation is distributed separately.
Choose the clip16 bundle for 16-frame clips or the clip32 bundle for 32-frame clips. Download both files from the selected bundle into the same directory, keeping their original filenames. With the compatible inference implementation, load the selected directory:
from tactile_jepa_inference import load_encoder
encoder = load_encoder("LingBot-Touch-Small-clip16")
# For the 32-frame variant:
# encoder = load_encoder("LingBot-Touch-Small-clip32")
The clip16 and clip32 variants were pretrained with clips of 16 and 32 frames, respectively. Both use a tubelet size of 4, so the input sequence length T must be a positive multiple of 4 during inference.
Input and Output
The encoder consumes normalized tactile signals with shape [B,T,5,M,3], canonical taxel coordinates, and a sensing-modality selection. The fixed finger order is thumb, index, middle, ring and pinky. Measurement X/Y are normalized to [-1,1] and Z to [0,1]. Sensor geometry, handedness, point ordering and fixed measurement ranges must be adapted correctly before inference.
The output consists of token-level latent representations and the corresponding temporal, finger and spatial addresses. These features can be used for downstream tactile perception and robotic manipulation; attention-based querying is recommended for task-specific readout.
License
The model bundles are distributed under the Apache License, Version 2.0.