- Add support for getting base64-encoded images via openAI's image_url.
- Add `context` as a parameter for the function `LogError`.
- Handle the error from `image.GetImageFromUrl` by logging it.
- Convert the role to `user` if it is `system` and add a dummy model message to make Gemini happy.
* fix: openai response should contains `model`
- Update model attributes in `claudeHandler` for `relay-claude.go`
- Implement model type for fullTextResponse in `relay-gemini.go`
- Add new `Model` field to `OpenAITextResponse` struct in `relay.go`
* chore: set model name response for models
---------
Co-authored-by: JustSong <songquanpeng@foxmail.com>
* Update relay-ali.go: 改进stream模式，添加联网搜索能力
通义千问支持stream的增量模式，不需要每次去掉上次的前缀；实测qwen-max联网模式效果不错，添加了联网模式。如果别的模型有问题可以改为单独给qwen-max开放
* 删除"stream参数"
刚发现原来阿里api没有这个参数，上次误加了。
* refactor: only enable search when specified
* fix: remove custom suffix when get model ratio
---------
Co-authored-by: JustSong <songquanpeng@foxmail.com>
- Update model attributes in `claudeHandler` for `relay-claude.go`
- Implement model type for fullTextResponse in `relay-gemini.go`
- Add new `Model` field to `OpenAITextResponse` struct in `relay.go`
```
- Refactor relay-text controller to improve readability and maintainability
- Optimize database queries for improved performance
- Update error handling to provide more informative messages
```
- Remove unused import `gorm.io/gorm/logger` from relay-text controller
- Refactor and consolidate related functions in relay-text controller
- Improve error handling in relay-text controller
- Refactor relay-text controller to improve performance and readability
- Update function signatures for better consistency
- Optimize error handling for improved reliability
- Update error logging in relay controller
- Include error summary and request details in error log message
- Improve formatting for more descriptive and comprehensive error log messages
* fest: Add Google Gemini Pro, fix#810
* fest: Add tooling to Gemini; Add OpenAI-like system prompt to Gemini
* refactor: removing unused if statement
* fest: Add dummy model message for system message in gemini model
* chore: update implementation
---------
Co-authored-by: JustSong <songquanpeng@foxmail.com>
- Refactored code in relay-utils.go
- Eliminated unused imports and redundant function
- Improved code readability with added comments
- Cleaned up by removing unnecessary commented-out code
* feat: Refactor response parsing logic to support multiple formats
The parsing logic for responses in relay.go and relay-audio.go was refactored to support multiple response formats - 'json', 'text', 'srt', 'verbose_json', and 'vtt'. The existing `WhisperResponse` struct was renamed to `WhisperJsonResponse` and a new struct `WhisperVerboseJsonResponse` was added to support the 'verbose_json' format. Additional parsing functions were added to extract text from these new response types. This change was necessary to make the parsing logic more flexible and extendable for different types of responses.
* chore: update name
---------
Co-authored-by: JustSong <songquanpeng@foxmail.com>
* fix: based on #754 add 'omitempty' in ImageRequest to fit official api reference for relay
* Revert "fix: based on #754 add 'omitempty' in ImageRequest to fit official api reference for relay"
This reverts commit b526006ce0.
* fix: add missing omitempty
---------
Co-authored-by: JustSong <songquanpeng@foxmail.com>
* fix: cloudflare test & expose detailed info about test failures
* fix: cloudflare test & expose detailed info about test failures
---------
Co-authored-by: JustSong <songquanpeng@foxmail.com>
* Updated ImageRequest struct and OpenAIModels,
added new Dall-E models and size ratios
* Fixed suspect `or`
* Refactored size ratio calculation in
relayImageHelper function
* Updated the format of resolution keys in
DalleSizeRatios map
* Added error handling for unsupported image size in
relayImageHelper function
* Added validation for number of generated images
and defined image generation ratios
* Refactored variable name from
DalleGenerationImageAmountRatios to
DalleGenerationImageAmounts
* Added validation for prompt length in
relayImageHelper function
* Updated model validation and removed size not
supported error in relayImageHelper function
* Refactored image size and model validation in
relayImageHelper function
* chore: discard binary file
* chore: update impl
---------
Co-authored-by: cktsun1031 <65409152+cktsun1031@users.noreply.github.com>
Co-authored-by: JustSong <songquanpeng@foxmail.com>
- Increase the length of the commit short SHA from 8 to 7 characters
- Update relay-aiproxy.go to handle multiple content types in the last message
- Refactor and modify countVisonTokenMessages function in relay-utils.go to handle different content types and update token counts accordingly
- Update VisionMessage struct in relay.go to have Content as a slice of OpenaiVisionMessageContent
- Modify error message in `VisionMessages` function to include failed JSON blob
- Include changes to omitted files in diff summary
- High-level summary of changes
- Add function `CountVisionImageToken` to count vision image tokens
- Modify function `imageSize` to handle different image types
- Add function `countVisonTokenMessages` to count tokens in vision messages
- Add logic to count tokens for different types of vision messages in `countVisonTokenMessages`
- Add tokens for role and name in `countVisonTokenMessages`
- Update total token count calculation in `countVisonTokenMessages` to include image tokens and message tokens
- Add constant values for tokens per message and tokens per name in `countVisonTokenMessages`
- Modify the error message on line 12 to include the JSON string that failed to unmarshal
- Added required packages to go.mod: `github.com/Laisky/errors/v2 v2.0.1`, `github.com/stretchr/testify v1.8.3`, `github.com/davecgh/go-spew v1.1.1`, `github.com/pmezard/go-difflib v1.0.0`
- Increased the number of returned recordings to 100 in `relay-utils.go`
- Refactored and simplified code in `relay-aiproxy.go` for improved message retrieval and error handling
- Added new test file `relay_test.go` and various test cases for different message types
- Modified functions in `group.go` and `relay.go` for improved functionality
- Added new dependencies: `github.com/fsnotify/fsnotify v1.4.9`, `github.com/go-playground/assert/v2 v2.2.0`, `github.com/nxadm/tail v1.4.8`, `github.com/onsi/ginkgo v1.16.5`, `github.com/onsi/gomega v1.18.1`, `golang.org/x/net v0.10.0`, `gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7`
- Updated dependencies: `github.com/gin-gonic/gin` from v1.9.0 to v2.0.0, `golang.org/x/net` from v0.17.0 to v0.10.0
- Removed dependencies: `github.com/golang-jwt/jwt v3.2.2+incompatible`, `github.com/gorilla/websocket v1.5.1`
- Updated Go version from `1.18` to `1.21`
- Made various modifications and refactoring in the code:
- Added new struct `VisionMessage` with fields `Role`, `Content`, and `Name`
- Added constants for certain types
- Added methods and error handling to handle different message types
- Modified existing struct and methods to accommodate changes
- Removed unused imports
