refactor(cache): improve caching system and add expiring cache #408
+301
−22
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
2 tasks
caarlos0
reviewed
Jan 6, 2025
|
hey! thanks for the PR! can you merge with main? thanks! |
With this approach, I was able to shave off ~1.2s in each request
… cache
Using a standard error type (os.ErrNotExist) instead of a custom error
message ("cache expired") makes the error handling more consistent with
Go conventions and allows for better error checking using errors.Is().
- Add error handling for file removal operations - Properly handle file.Close() errors using defer functions
- Add package-level documentation for cache types and constants - Document public methods and types with descriptive comments - Reorganize constant and variable declarations
nathabonfim59
force-pushed
the
cache.copilot
branch
from
473e6a4 to
9cb7116
Compare
- Use os.ModePerm constant for directory creation - Add clarifying constant for filename format validation - Fix documentation periods consistency - Improve code spacing
- Add descriptive error messages with wrapping - Use context in HTTP requests - Improve error handling in HTTP responses
nathabonfim59
force-pushed
the
cache.copilot
branch
from
f10dfb3 to
f2bcdd5
Compare
|
Done! Sorry for the Had some conflicts with main in my fork and had to rebase my commits to make it work. Also added a wrapper around the errors and fixed some minor linting issues. If you need me to change anything, just let me know. |
caarlos0
reviewed
Jan 9, 2025
| } | ||
|
|
||
| func (c *convoCache) read(id string, messages *[]openai.ChatCompletionMessage) error { | ||
| func (c *Cache[T]) cacheDir() string { |
There was a problem hiding this comment.
Suggested change
| func (c *Cache[T]) cacheDir() string { | |
| func (c *Cache[T]) dir() string { |
no nead to repeat cache I think
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
For my usage which primarily uses Copilot, this change made a huge difference in performance - reducing about ~2 seconds for each request by caching the access tokens.
Not sure if this is the direction you want to go, but I found it very helpful for my workflow.
Let me know if you'd like any changes - happy to update!
Changes
Convert existing cache to generic Cache[T] type
Add ExpiringCache implementation
Note: This PR builds on #406 and should be merged after it.