0.2.0

What's Changed

Happy new year! This release contains a significant overhaul of many elements of guidance, including:

• Visualization Overhaul: Guidance's in-line visualizations have been rewritten from scratch. TL;DR: We added a lot of code to support better model metrics and visualization. Check it out:

• Significantly Faster, Rust-based Grammar Engine: Guidance's core grammar processing has been fully migrated to the llguidance Rust library. Our grammar processing is now significantly faster -- state of the art across frameworks -- and fixes some key, subtle bugs in the earlier processing engine. All constrained decoding code will now run much faster.

• Increased JSON schema coverage: We've spent a lot of time focusing on JSON schema support in guidance, and now have leading coverage of JSON schema constraint types via guidance's json function.

• And lots more general quality of life improvements! See below for full list of PRs and changelogs. Thank you to all the existing and new contributors that made this possible!

---

• [WIP] Replace EarleyParser with lexeme-based rust implementation by @hudson-ai in #951
• Update README.md with new grammar-enabled endpoint details (Azure AI) by @Harsha-Nori in #998
• [Test] Enable Azure Guidance endpoints in test matrix by @riedgar-ms in #999
• [Test] Move Azure Guidance in CI by @riedgar-ms in #1000
• [Bug] del f._self_call_placeholder_ even when calling f raises exception by @hudson-ai in #994
• [mypy] Improve types in _grammar.py by @riedgar-ms in #1002
• [Build] CI & Notebook Workaround by @riedgar-ms in #1005
• [Feature] Allow oneOf in JSON schemas (with limited support) by @hudson-ai in #982
• Use json_string kwarg from llguidance 0.1.7to properly escape json string patterns by @hudson-ai in #1008
• Remove Placeholder and replace_grammar_node in favor of pointer-like container object by @hudson-ai in #1007
• [Feature] Support required properties in JSON schemas by @hudson-ai in #1009
• [Housekeeping] Clean up Enums to make JSON features more transparent by @hudson-ai in #1011
• [Doc] Fix typos in README and tutorials notebooks by @gfesatidis in #1012
• fix test_ll for new llguidance with working advance_parser() by @mmoskal in #1013
• [Bug] Ensure grammar.match raises appropriately on incomplete input by @hudson-ai in #1016
• Support boolean JSON schemas by @hudson-ai in #1015
• @overload the guidance decorator in order to improve static type-checking by @hudson-ai in #1014
• JSON string 'format' kwarg by @hudson-ai in #984
• Strip anchors from JSON regexes by @hudson-ai in #1027
• JSON reserved keywords by @hudson-ai in #1032
• "required" JSON keys are validated against "additionalProperties" if they are missing from "properties" by @hudson-ai in #1029
• [Bug] Make _self_call_placeholder_ thread-safe by @hudson-ai in #1034
• add test for "nullable bug" by @mmoskal in #1025
• Make the tests pass with larger tokenizers by @mmoskal in #1037
• [Feature] Support for JSON minimum , maximum, exclusiveMinimum, and exclusiveMaximum by @hudson-ai in #1023
• [Feature] Allow @guidance to decorate methods by @hudson-ai in #1035
• [Bug] [Feature] Fix issue #1039 by exposing And, Not regex operators from llguidance by @hudson-ai in #1043
• [WIP] Try to get PR Gate working again... by @hudson-ai in #1048
• [Feature] Enable user-passed separators keyword to guidance.json by @hudson-ai in #1044
• fix typos in README by @Saibo-creator in #1056
• [Feature] Expand JSON $ref resolution by @hudson-ai in #1061
• [JSON] Raise exceptions when "sibling" keywords are unhandled by @hudson-ai in #1063
• [Feature] Allow json-loads-able strings to be passed as schema by @hudson-ai in #1028
• [Hotfix] Echo forced to false on terminal detection. by @nopdive in #1067
• [Keeping House] Split JSON tests into multiple files by @hudson-ai in #1072
• [Feature] [JSON] Broader support of allOf JSON keyword by @hudson-ai in #1070
• [BUG] Reset role blocks when model.reset is called by @hudson-ai in #1073
• [Optimization] Advance parser concurrently with model forward pass by @hudson-ai in #1065
• Update README.md by @culmat in #1045
• [Feature] Add qwen2.5 chat templates by @xruifan in #1066
• Expose fastforward and backtrack llguidance flags by @Harsha-Nori in #1075
• [Bug] Fix for environment detection in barebones environments. by @nopdive in #1076
• Update setup.py to limit llguidance version by @hudson-ai in #1080
• update to llguidance 0.4.1 by @mmoskal in #1081
• fix current failures in pytest when run locally by @paulbkoch in #1083
• Force Transformers engine to return float logits by @JC1DA in #1086
• Bump runners from macos-12 to macos-14 by @hudson-ai in #1088
• update llguidance to 0.5.0 by @mmoskal in #1087
• Use llguidance JSON grammar by @hudson-ai in #1085
• Upgrade llguidance to 0.5.1rc0 by @hudson-ai in #1090
• Interactivity Overhaul (User Interface & Model Instrumentation & Network Comms) by @nopdive in #1054

New Contributors

• @gfesatidis made their first contribution in #1012
• @Saibo-creator made their first contribution in #1056
• @culmat made their first contribution in #1045
• @xruifan made their first contribution in #1066
• @JC1DA made their first contribution in #1086

Full Changelog: 0.1.16...0.2.0

0.2.0rc1

What's Changed

• Replace Python-based EarleyParser with lexeme-based rust implementation by @hudson-ai & @mmoskal in #951

This pre-release version of guidance uses a new Rust-based Earley lexer + parser that offers significant speedups and bugfixes to guidance grammar parsing. See PR #951 for full details of the optimizations. Expect some changes from current Guidance (v0.1.16) behavior during pre-release.

Full Changelog: 0.1.16...0.2.0rc1

0.1.16

What's Changed

• Allow generating arbitrary (schemaless) JSON by @hudson-ai in #892
• [Bug] Fix Llama3ChatTemplate by @jjkjkj in #884
• Use python's re module instead of pyformlang to build regex ASTs by @hudson-ai in #854
• [Feature] Add Support for Phi-3-small by @ambisinister in #935
• [Feature] Constant elements in JSON schemas by @wjn0 in #886
• [Bug] Fix for LlamaCpp tokeniser prepending spaces by @riedgar-ms in #903
• New Benchmarking system by @nopdive in #843
• Mistral chat template integrated by @Harsha-Nori in #919
• [Bug] Passing through kwargs to TransformersTokenizer by @riedgar-ms in #922
• [Feature] Add extra support to JSON string schema by @riedgar-ms in #927
• Update protobuf definitions for remote list_append fix by @nking-1 in #937
• Downgrade protobuf required version using protoc 25.3 by @nking-1 in #946
• [Bug] Fix regex for urls in image function by @kklemon in #928
• Dedent fixes for Python 3.12 by @Harsha-Nori in #959
• Refector TransformersTokenizer and change fallback behavior for byte_tokens by @hudson-ai in #973
• Fix Transformer models with bos_token null by @Mihaiii in #990

New Contributors

• @nopdive made their first contribution in #843
• @jjkjkj made their first contribution in #884
• @wjn0 made their first contribution in #886
• @kklemon made their first contribution in #928
• @ambisinister made their first contribution in #935
• @Mihaiii made their first contribution in #990

Full Changelog: 0.1.15...0.1.16

0.1.15

What's Changed

• Support for Anthropic Claude 3 and Phi-3 models by @Harsha-Nori
• [Feature] Metrics for Grammarless models by @riedgar-ms in #817
• Remove Instruct/Chat versions of models & introduce a new ChatTemplate API by @Harsha-Nori in #820
• Fix set_attribute by @slundberg in #835
• Fix for unstable tokenization schemes by @Harsha-Nori in #830
• Support null captures (Fix #826) by @slundberg in #834
• Allow capture of raw functions by @slundberg in #837
• [Build] Significant notebook testing updates by @riedgar-ms in #825
• [Docs] ReadTheDocs Now Works! by @riedgar-ms in #829
• add initial support for azure guidance server API by @paulbkoch in #827
• [Build] Adjust llama-cpp-python versions by @riedgar-ms in #832
• [Test] Fix for LlamaCpp tests by @paulbkoch in #839
• Remove default max token limit by @slundberg in #836
• [Build] General Maintenance by @riedgar-ms in #842
• Updated readme by @nking-1 in #810

Full Changelog: 0.1.14...0.1.15

0.1.14

This release brings a series of new features, bug fixes, and changes, including:

• Improved compatibility for loading many transformers models, including newly tested support for Phi-3
• New capabilities in the guidance.json function for generating json, including early support for pydantic schemas.
• Support for AzureAI models in new guidance.models.AzureAIStudio* classes
• Significant dependency reductions on the default guidance installation
• Small grammar optimizations
• Many improvements to the infrastructure, testing, and release pipelines :)

What's Changed

• replace catches of the ImportError exception with the more specific ModuleNotFoundError exception by @paulbkoch in #733
• fix for issue #660 where the call to llama_batch_free is failing during process shutdown by @paulbkoch in #734
• Fix #713 for mistral models in transformers by @slundberg in #740
• Update JSON API and example notebook by @riedgar-ms in #699
• remove redundant options from select() by @mmoskal in #712
• Add support for prefixItems, minItems, and maxItems for JSON array generation by @hudson-ai in #732
• [Docs] Expand documentation by @riedgar-ms in #750
• Generation with pydantic schemas by @hudson-ai in #724
• First stab at minimal mypy implementation by @ryanpeach in #738
• remove openai hard dependency by @slundberg in #763
• from guidance import guidance by @ryanpeach in #759
• Run black on codebase by @Harsha-Nori in #765
• add rust bindings by @paulbkoch in #736
• fix Transformers tokenizer issues when models have extended vocabulary and/or special tokens by @Harsha-Nori in #766
• [Install] Rework optional imports by @riedgar-ms in #767
• Update _llama_cpp.py based on new contract of llama_get_logits by @czoccolo in #769
• [Bug] Missing support for AzureAI by @riedgar-ms in #770
• [Doc] Add developer notes by @riedgar-ms in #774
• [Feature] Improved JSON Schema support by @riedgar-ms in #787
• [Bug] Fix Regex for identifying OpenAI models by @alexander-brady in #778
• Fix models like phi-3 which have a mismatched tokenizer definition and model output tensor size by @Harsha-Nori in #792
• [Feature] Add temperature to JSON generator by @riedgar-ms in #784
• [Feature] Support for Azure AI Studio by @riedgar-ms in #779
• Add guard for running build from setup.py, redirecting users to pip by @Harsha-Nori in #805
• Many testing and CI improvements by @riedgar-ms

New Contributors

• @mmoskal made their first contribution in #712
• @ryanpeach made their first contribution in #738
• @czoccolo made their first contribution in #769
• @alexander-brady made their first contribution in #778

Full Changelog: 0.1.13...0.1.14

0.1.13

What's Changed

• Fixed PyPi publishing
• Workflow for GPU Runner by @riedgar-ms in #694
• Add Llama to model test matrix by @riedgar-ms in #703

Full Changelog: 0.1.12...0.1.13

0.1.12

Lots of new features and bug fixes :)

What's Changed

• Adding Caching to OpenAI chat by @adamgordonbell in #603
• Minor Server Tweaks by @riedgar-ms in #612
• Dependency trimming by @riedgar-ms in #613
• Add Python 3.12 by @riedgar-ms in #623
• Grammar Builder for JSON Schema by @riedgar-ms in #619
• remove unused code by @paulbkoch in #629
• fix: missing tokenizer choice in AzureOpenAI subclasses by @imarquart in #641
• Pop first argument from guidance wrapper by @hudson-ai in #634
• Fix behavior of _json_schema_to_grammar for schemas with many properties by @hudson-ai in #645
• Exclude tests packages from distribution by @shawnz in #646
• Support multiple additions to ModelStream by @shawnz in #639
• Enable AzureOpenAI model testing by @riedgar-ms in #514
• CI Build Schedule by @riedgar-ms in #662
• Add extensible tests to transformers by @riedgar-ms in #663
• Update _llama_cpp.py to use new bindings by @Nking92 in #665
• Propagate errors in ModelStream by @shawnz in #640
• Fix test_transformers.py by @riedgar-ms in #673
• Together.ai Support by @minosvasilias in #654
• Move get_logits and EngineCallResponse out of the Engine.call function so that the remaining parts can be lowered to Rust or C++ in the future and for simplification of LLM servers that operate in batches by @paulbkoch in #647
• Make Nulls Terminal by @hudson-ai in #679
• [Test] Expand to other OSes by @riedgar-ms in #680
• JSON Schema Rewrite by @riedgar-ms in #651
• [Test] MacOS Issue with Server tests by @riedgar-ms in #684
• Support mutual recursion in GrammarFunctions by @hudson-ai in #683
• More JSON schema features by @hudson-ai in #685
• Improve negative test cases for JSON generation by @riedgar-ms in #693
• Separate out server tests by @riedgar-ms in #688
• Add fixture for an LLM by @riedgar-ms in #674
• Removed side effect on the stop argument from _gen by @VincentToups in #677
• Fix for test_gen by @riedgar-ms in #695
• Fix #681 and #682 by @slundberg in #704

New Contributors

• @adamgordonbell made their first contribution in #603
• @imarquart made their first contribution in #641
• @hudson-ai made their first contribution in #634
• @shawnz made their first contribution in #646
• @Nking92 made their first contribution in #665
• @minosvasilias made their first contribution in #654
• @VincentToups made their first contribution in #677

Full Changelog: 0.1.11...0.1.12

0.1.11

What's Changed

• Fix GPU usage by removing device from Transformers class wrapper to use the device/device_map directly exposed by HF Transformers in kwargs by @cpcdoy in #569
• Adding streaming support under guidance v0.1 by @hodlen in #568
• Fix typos in intro_to_guidance.ipynb by @trungsudo in #574
• Update "Basic Tutorial" link in README.md by @alexandremuzio in #580
• Prompt role blocks debug output + new prompt implementation example by @cpcdoy in #578
• WIP: fix: AzureOpenAI authentication to support API Key auth by @younes-io in #583
• Allow client/server guidance execution by @slundberg in #586

New Contributors

• @cpcdoy made their first contribution in #569
• @hodlen made their first contribution in #568
• @trungsudo made their first contribution in #574
• @alexandremuzio made their first contribution in #580
• @younes-io made their first contribution in #583

Full Changelog: 0.1.10...0.1.11

0.1.10

0.1.9 was missing the hugging face patch...so 0.1.10 is real 0.1.9 :)

Full Changelog: 0.1.9...0.1.10

0.1.9

What's Changed

• Bug patch for HuggingFace transformers tokenization issue.
• Fixed generation coloring for multiple sequence generator calls.
• Bug fix for the base AzureOpenAI class by @riedgar-ms in #543
• Google AI Studio's Gemini by @Anilturaga in #550
• add react notebook by @Sam1320 in #547

Full Changelog: 0.1.8...0.1.9