Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(core): Implement stop generation and state management #34

Open
inidal opened this issue Jan 10, 2025 · 0 comments
Open

feat(core): Implement stop generation and state management #34

inidal opened this issue Jan 10, 2025 · 0 comments

Comments

@inidal
Copy link

inidal commented Jan 10, 2025

Current Behavior

  1. Long text generation sometimes stops unexpectedly
  2. After app restart, error appears: "Platform error: Previous invocation still processing. Wait for done=true"
  3. No way to manually stop ongoing generation

Expected Behavior

  1. Text generation should complete fully or be manually stoppable
  2. App should handle previous generation state on restart
  3. Add ability to stop generation manually

Proposed Solutions

  1. Add stopGeneration() method to allow manual interruption
  2. Implement proper cleanup of generation state on app lifecycle changes
  3. Add state management to track and reset generation status

Technical Details

Affected components:

  • FlutterGemmaPlugin (Kotlin)
  • InferenceModel (Kotlin)
  • flutter_gemma_interface.dart
  • flutter_gemma_mobile.dart

The LLMInference API from MediaPipe should support stopping generation - this could be exposed through the plugin interface.

Impact

This would improve:

  • User experience by allowing generation cancellation
  • App stability by preventing hanging states
  • Error handling for interrupted generations

Additional Context

This is particularly important for mobile apps where users may want to stop long generations or switch away from the app mid-generation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant