implemented v1.4.0

Gesture System Overview

Overview

CleverKeys implements a multi-layered gesture recognition system that handles four gesture types: short swipes (directional swipes within a key for sublabels), long swipes (gestures across keys for neural word prediction), circle/rotation gestures (for double letters), and slider gestures (continuous value adjustment). The hasLeftStartingKey flag is the central decision point that routes touches to the appropriate handler.

This spec is the system-level overview. Per-gesture behavior is documented in the linked specifications under “Related Specifications”.

Key Files

File	Class/Function	Purpose
`src/main/kotlin/tribixbite/cleverkeys/Pointers.kt`	`Pointers`	Touch event handling, gesture pipeline routing (~1789 lines)
`src/main/kotlin/tribixbite/cleverkeys/Pointers.kt`	`Pointers.Pointer` (internal class)	Per-pointer state including `hasLeftStartingKey`
`src/main/kotlin/tribixbite/cleverkeys/GestureClassifier.kt`	`GestureClassifier`	TAP vs SWIPE classification (65 lines)
`src/main/kotlin/tribixbite/cleverkeys/Gesture.kt`	`Gesture`	Circle/rotation state machine (141 lines)
`src/main/kotlin/tribixbite/cleverkeys/customization/SwipeDirection.kt`	`SwipeDirection`	8-direction enum for short swipes
`src/main/kotlin/tribixbite/cleverkeys/Config.kt`	Gesture settings	Configuration thresholds
`src/main/kotlin/tribixbite/cleverkeys/Keyboard2View.kt`	`getKeyHypotenuse()`	Key dimension calculation

Architecture

Touch Events (Keyboard2View.onTouchEvent)
           │
           ▼
      Pointers.kt
           │
     ┌─────┴─────┐
     │           │
  onTouchMove  onTouchUp
     │           │
     ▼           ▼
┌─────────┐  ┌──────────────────┐
│ Track   │  │ GestureClassifier │
│ hasLeft │  │    .classify()    │
│ Starting│  └────────┬─────────┘
│ Key     │           │
└─────────┘    ┌──────┴──────┐
               │             │
          SWIPE           TAP
               │             │
               ▼             ▼
        Neural Predictor  Short Gesture
        (onSwipeEnd)      Handler

Data Flow

Touch Tracking

onTouchDown: Record start position, identify starting key
onTouchMove: Update position, check if left starting key
onTouchUp: Classify gesture, trigger appropriate handler

The `hasLeftStartingKey` Gatekeeper

This boolean flag is the single decision point determining gesture type:

// Pointers.kt, onTouchMove handler
if (ptr.key != null && !ptr.hasLeftStartingKey) {
    val keyHypotenuse = _handler.getKeyHypotenuse(ptr.key)
    val maxAllowedDistance = keyHypotenuse * (config.short_gesture_max_distance / 100.0f)
    val distanceFromStart = sqrt((x - ptr.downX).pow(2) + (y - ptr.downY).pow(2))

    if (distanceFromStart > maxAllowedDistance) {
        ptr.hasLeftStartingKey = true  // Permanently set for this touch
    }
}

See Pointers.kt:734-744 for the live implementation.

Configuration

Key	Type	Default	Description
`short_gesture_min_distance`	Int	15	Min pixels for short swipe detection
`short_gesture_max_distance`	Int	50	Max % of key hypotenuse for short swipe (above = long swipe)
`tap_duration_threshold`	Long	200	Max ms for tap vs swipe distinction
`swipe_speed_threshold`	Float	0.4	Min speed for swipe typing activation
`circle_gesture_enabled`	Boolean	true	Enable circle gestures for double letters

Public API

GestureClassifier

class GestureClassifier(private val context: Context) {

    enum class GestureType { TAP, SWIPE }

    data class GestureData(
        val hasLeftStartingKey: Boolean,
        val totalDistance: Float,
        val timeElapsed: Long,
        val keyWidth: Float
    )

    fun classify(gesture: GestureData): GestureType {
        val minSwipeDistance = gesture.keyWidth / 2.0f

        return if (gesture.hasLeftStartingKey &&
                   (gesture.totalDistance >= minSwipeDistance ||
                    gesture.timeElapsed > maxTapDurationMs)) {
            GestureType.SWIPE
        } else {
            GestureType.TAP
        }
    }
}

Pointers Touch Handlers

class Pointers(
    private val handler: Handler,
    private val config: Config
) {
    // Called from Keyboard2View.onTouchEvent
    fun onTouchEvent(event: MotionEvent): Boolean

    // Internal handlers
    private fun onTouchDown(ptr: Pointer, x: Float, y: Float)
    private fun onTouchMove(ptr: Pointer, x: Float, y: Float)
    private fun onTouchUp(ptr: Pointer)

    // Gesture routing
    private fun handleShortGesture(ptr: Pointer, direction: SwipeDirection)
    private fun handleSwipeTyping(ptr: Pointer)
}

Implementation Details

Gesture Classification Logic

hasLeftStartingKey	Distance	Time	Result
FALSE	any	any	TAP
TRUE	< keyWidth/2	<= tap_duration	TAP
TRUE	>= keyWidth/2	any	SWIPE
TRUE	any	> tap_duration	SWIPE

Key Dimension Calculation

All thresholds use actual device pixels computed at runtime:

// Keyboard2View.kt
override fun getKeyHypotenuse(key: KeyboardData.Key): Float {
    val tc = themeComputed ?: return 0f

    // Find row height from layout
    var normalizedRowHeight = 0f
    for (row in keyboard.rows) {
        for (k in row.keys) {
            if (k == key) {
                normalizedRowHeight = row.height
                break
            }
        }
    }

    // Convert to actual pixels
    val keyHeightPx = normalizedRowHeight * tc.row_height
    val keyWidthPx = key.width * keyWidth

    return sqrt(keyWidthPx.pow(2) + keyHeightPx.pow(2))  // Diagonal in pixels
}

Short Swipe Direction Detection

Direction calculated from delta between start and end positions:

private fun calculateSwipeDirection(dx: Float, dy: Float): SwipeDirection {
    val angle = atan2(-dy.toDouble(), dx.toDouble())  // Negative Y because screen coords
    val degrees = Math.toDegrees(angle)

    return when {
        degrees in -22.5..22.5 -> SwipeDirection.E
        degrees in 22.5..67.5 -> SwipeDirection.NE
        degrees in 67.5..112.5 -> SwipeDirection.N
        degrees in 112.5..157.5 -> SwipeDirection.NW
        degrees > 157.5 || degrees < -157.5 -> SwipeDirection.W
        degrees in -157.5..-112.5 -> SwipeDirection.SW
        degrees in -112.5..-67.5 -> SwipeDirection.S
        degrees in -67.5..-22.5 -> SwipeDirection.SE
        else -> SwipeDirection.E
    }
}

Circle Gesture Detection

Circle gestures detected via rotation accumulation:

class Gesture {
    private var totalRotation: Float = 0f
    private var lastAngle: Float = 0f

    fun addPoint(x: Float, y: Float, centerX: Float, centerY: Float) {
        val currentAngle = atan2(y - centerY, x - centerX)
        val delta = normalizeAngle(currentAngle - lastAngle)
        totalRotation += delta
        lastAngle = currentAngle
    }

    fun isCircleComplete(): Boolean {
        return abs(totalRotation) >= 2 * PI  // Full circle (360 degrees)
    }

    fun getDirection(): CircleDirection {
        return if (totalRotation > 0) CircleDirection.CLOCKWISE
               else CircleDirection.COUNTER_CLOCKWISE
    }
}

Swipe Typing Activation

Long swipes trigger neural prediction when finger leaves starting key:

private fun onTouchUp(ptr: Pointer) {
    val gestureType = gestureClassifier.classify(GestureData(
        hasLeftStartingKey = ptr.hasLeftStartingKey,
        totalDistance = ptr.totalDistance,
        timeElapsed = ptr.timeElapsed,
        keyWidth = getKeyWidth(ptr.key)
    ))

    when (gestureType) {
        GestureType.SWIPE -> {
            if (ptr.hasLeftStartingKey) {
                // Long swipe - neural prediction
                onSwipeEnd(ptr.swipePath)
            } else {
                // Short swipe - sublabel action
                val direction = calculateSwipeDirection(ptr.dx, ptr.dy)
                handleShortGesture(ptr, direction)
            }
        }
        GestureType.TAP -> {
            handleTap(ptr.key)
        }
    }
}

Pointer State

The per-pointer state lives in the internal Pointers.Pointer class (Pointers.kt:1389). The conceptual shape:

data class Pointer(
    var key: KeyboardData.Key?,       // Starting key
    var downX: Float,                  // Initial touch X
    var downY: Float,                  // Initial touch Y
    var currentX: Float,               // Current X
    var currentY: Float,               // Current Y
    var downTime: Long,                // Touch start time
    var hasLeftStartingKey: Boolean,   // The gatekeeper flag
    var swipePath: MutableList<Point>, // Path for neural prediction
    var flags: Int                     // State flags (trackpoint, selection-delete, etc.)
)

Short Swipes - Per-key 8-direction sublabel gestures
Circle Gestures - Rotation-based double-letter input
Cursor Navigation - Spacebar swipe cursor movement
Selection / Delete - Backspace swipe word-delete and selection
Trackpoint Mode - Continuous cursor control
Neural Prediction - The long-swipe consumer
Swipe Typing - User-facing swipe typing feature

All specs User wiki Report doc issue