13 Commits

Author SHA1 Message Date
Pijus Kamandulis
26dcd68ace Added Docker to releaser configuration 2024-04-06 19:07:14 +03:00
Pijus Kamandulis
86c0275410 Added ability to configure using environment variables 2024-03-11 23:35:47 +02:00
Pijus Kamandulis
398584368f Implement OFFSET LIMIT 2024-03-11 22:09:55 +02:00
Pijus Kamandulis
5b66828bd0 Added some docs 2024-03-11 20:47:44 +02:00
Pijus Kamandulis
6ed74688ca Implement AVG, COUNT, MAX, MIN, SUM functions 2024-03-11 19:10:41 +02:00
Pijus Kamandulis
b72bba86c8 Implement 'GROUP BY' statement 2024-03-11 17:50:20 +02:00
Pijus Kamandulis
18edb925bf Added instructions for installing using Homebrew 2024-02-27 22:46:13 +02:00
Pijus Kamandulis
6ccb7c4bdd Implement custom logger with log levels 2024-02-27 22:38:59 +02:00
Pijus Kamandulis
b9e38575bc Load state from '-Persist' path if '-InitialData' not supplied 2024-02-27 22:11:33 +02:00
Pijus Kamandulis
3aeae98404 Added pre-generated TLS certificate 2024-02-27 21:58:57 +02:00
Pijus Kamandulis
5ff923ce2c Implement DISTINCT clause 2024-02-27 21:10:03 +02:00
Pijus Kamandulis
f3f3966dd5 Fix server info response 2024-02-27 20:27:51 +02:00
Pijus Kamandulis
19f62f8173 Fix partition key ranges endpoint 2024-02-27 20:08:48 +02:00
36 changed files with 3010 additions and 1051 deletions

14
.github/ISSUE_TEMPLATE.md vendored Normal file
View File

@@ -0,0 +1,14 @@
#### Summary
Bug report in one concise sentence
#### Steps to reproduce
How can we reproduce the issue (what version are you using?)
#### Expected behavior
Describe your issue in detail
#### Observed behavior (that appears unintentional)
What did you see happen? Please include relevant error messages.
#### Possible fixes
If you can, link to the line of code that might be responsible for the problem

18
.github/PULL_REQUEST_TEMPLATE.md vendored Normal file
View File

@@ -0,0 +1,18 @@
<!-- Thank you for contributing a pull request! Here are a few tips to help you:
1. If applicable, please check if unit tests are added for new features
2. Read the contribution guide lines https://github.com/pikami/cosmium/docs/CONTRIBUTING.md
-->
#### Summary
<!--
A description of what this pull request does, as well as QA test steps (if applicable).
-->
#### Ticket Link
<!--
If applicable, please include a link to the GitHub issue:
Fixes https://github.com/pikami/cosmium/issues/XXX
-->

View File

@@ -7,6 +7,7 @@ on:
permissions:
contents: write
packages: write
jobs:
goreleaser:
@@ -20,6 +21,12 @@ jobs:
uses: actions/setup-go@v5
with:
go-version: 1.21.6
- name: Docker Login
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Run GoReleaser
uses: goreleaser/goreleaser-action@v5
with:

View File

@@ -26,10 +26,25 @@ brews:
homepage: 'https://github.com/pikami/cosmium'
repository:
owner: pikami
name: homebrew-pikami
name: homebrew-brew
commit_author:
name: pikami
email: git@pikami.org
dockers:
- image_templates: ["ghcr.io/pikami/{{ .ProjectName }}:{{ .Version }}-amd64"]
dockerfile: Dockerfile
use: docker
build_flag_templates:
- "--platform=linux/amd64"
- "--pull"
- "--label=org.opencontainers.image.title={{.ProjectName}}"
- "--label=org.opencontainers.image.description=Lightweight Cosmos DB emulator"
- "--label=org.opencontainers.image.url=https://github.com/pikami/cosmium"
- "--label=org.opencontainers.image.source=https://github.com/pikami/cosmium"
- "--label=org.opencontainers.image.created={{.Date}}"
- "--label=org.opencontainers.image.revision={{.FullCommit}}"
- "--label=org.opencontainers.image.version={{.Version}}"
checksum:
name_template: 'checksums.txt'

6
Dockerfile Normal file
View File

@@ -0,0 +1,6 @@
FROM scratch
WORKDIR /app
COPY cosmium /app/cosmium
ENTRYPOINT ["/app/cosmium"]

View File

@@ -1,11 +1,21 @@
# Cosmium
Cosmium is a lightweight Cosmos DB emulator designed to facilitate local development and testing. While it aims to provide developers with a solution for running a local database during development, it's important to note that it's not 100% compatible with Cosmos DB. However, it serves as a convenient tool for E2E or integration tests during the CI/CD pipeline.
Cosmium is a lightweight Cosmos DB emulator designed to facilitate local development and testing. While it aims to provide developers with a solution for running a local database during development, it's important to note that it's not 100% compatible with Cosmos DB. However, it serves as a convenient tool for E2E or integration tests during the CI/CD pipeline. Read more about compatibility [here](docs/compatibility.md).
One of Cosmium's notable features is its ability to save and load state to a single JSON file. This feature makes it easy to load different test cases or share state with other developers, enhancing collaboration and efficiency in development workflows.
# Getting Started
### Downloading Cosmium
### Installation via Homebrew
You can install Cosmium using Homebrew by adding the `pikami/brew` tap and then installing the package.
```sh
brew tap pikami/brew
brew install cosmium
```
This will download and install Cosmium on your system, making it easy to manage and update using Homebrew.
### Downloading Cosmium Binaries
You can download the latest version of Cosmium from the [GitHub Releases page](https://github.com/pikami/cosmium/releases). Choose the appropriate release for your operating system and architecture.
@@ -23,11 +33,7 @@ Cosmium is available for the following platforms:
Once downloaded, you can launch Cosmium using the following command:
```sh
./cosmium-linux-amd64 \
-Cert "cert.crt" \
-CertKey "cert.key" \
-Persist "./save.json" \
-InitialData "./save.json"
cosmium -Persist "./save.json"
```
Connection String Example:
@@ -41,9 +47,22 @@ If you want to run Cosmos DB Explorer alongside Cosmium, you'll need to build it
Once running, the explorer can be reached by navigating following URL: `https://127.0.0.1:8081/_explorer/` (might be different depending on your configuration).
### Running with docker (optional)
If you wan to run the application using docker, configure it using environment variables see example:
```sh
docker run --rm \
-e Persist=/save.json \
-v ./save.json:/save.json \
-p 8081:8081 \
ghcr.io/pikami/cosmium
```
### SSL Certificate
By default, Cosmium runs on HTTP. However, if you provide an SSL certificate, it will use HTTPS. Most applications will require HTTPS, so you can specify paths to the SSL certificate and key (PEM format) using the `-Cert` and `-CertKey` arguments, respectively.
By default, Cosmium uses a pre-generated SSL certificate. You can provide your own certificates by specifying paths to the SSL certificate and key (PEM format) using the `-Cert` and `-CertKey` arguments, respectively.
To disable SSL and run Cosmium on HTTP instead, you can use the `-DisableTls` flag. However most applications will require HTTPS.
### Other Available Arguments
@@ -51,10 +70,20 @@ By default, Cosmium runs on HTTP. However, if you provide an SSL certificate, it
* **-DisableAuth**: Disable authentication
* **-Host**: Hostname (default "localhost")
* **-InitialData**: Path to JSON containing initial state
* **-Persist**: Saves data to the given path on application exit
* **-Persist**: Saves data to the given path on application exit (When `-InitialData` argument is not supplied, it will try to load data from path supplied in `-Persist`)
* **-Port**: Listen port (default 8081)
* **-Debug**: Runs application in debug mode, this provides additional logging
These arguments allow you to configure various aspects of Cosmium's behavior according to your requirements.
All mentioned arguments can also be set using environment variables:
* **COSMIUM_ACCOUNTKEY** for `-AccountKey`
* **COSMIUM_DISABLEAUTH** for `-DisableAuth`
* **COSMIUM_HOST** for `-Host`
* **COSMIUM_INITIALDATA** for `-InitialData`
* **COSMIUM_PERSIST** for `-Persist`
* **COSMIUM_PORT** for `-Port`
* **COSMIUM_DEBUG** for `-Debug`
# License
This project is [MIT licensed](./LICENSE).

View File

@@ -3,10 +3,13 @@ package config
import (
"flag"
"fmt"
"os"
"strings"
)
const (
DefaultAccountKey = "C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw=="
EnvPrefix = "COSMIUM_"
)
var Config = ServerConfig{}
@@ -20,9 +23,12 @@ func ParseFlags() {
initialDataPath := flag.String("InitialData", "", "Path to JSON containing initial state")
accountKey := flag.String("AccountKey", DefaultAccountKey, "Account key for authentication")
disableAuthentication := flag.Bool("DisableAuth", false, "Disable authentication")
disableTls := flag.Bool("DisableTls", false, "Disable TLS, serve over HTTP")
persistDataPath := flag.String("Persist", "", "Saves data to given path on application exit")
debug := flag.Bool("Debug", false, "Runs application in debug mode, this provides additional logging")
flag.Parse()
setFlagsFromEnvironment()
Config.Host = *host
Config.Port = *port
@@ -32,9 +38,25 @@ func ParseFlags() {
Config.InitialDataFilePath = *initialDataPath
Config.PersistDataFilePath = *persistDataPath
Config.DisableAuth = *disableAuthentication
Config.DisableTls = *disableTls
Config.Debug = *debug
Config.DatabaseAccount = Config.Host
Config.DatabaseDomain = Config.Host
Config.DatabaseEndpoint = fmt.Sprintf("https://%s:%d/", Config.Host, Config.Port)
Config.AccountKey = *accountKey
}
func setFlagsFromEnvironment() (err error) {
flag.VisitAll(func(f *flag.Flag) {
name := EnvPrefix + strings.ToUpper(strings.Replace(f.Name, "-", "_", -1))
if value, ok := os.LookupEnv(name); ok {
err2 := flag.Set(f.Name, value)
if err2 != nil {
err = fmt.Errorf("failed setting flag from environment: %w", err2)
}
}
})
return
}

View File

@@ -14,4 +14,6 @@ type ServerConfig struct {
InitialDataFilePath string
PersistDataFilePath string
DisableAuth bool
DisableTls bool
Debug bool
}

View File

@@ -1,13 +1,13 @@
package middleware
import (
"fmt"
"net/url"
"strings"
"github.com/gin-gonic/gin"
"github.com/pikami/cosmium/api/config"
"github.com/pikami/cosmium/internal/authentication"
"github.com/pikami/cosmium/internal/logger"
)
func Authentication() gin.HandlerFunc {
@@ -53,7 +53,7 @@ func Authentication() gin.HandlerFunc {
params, _ := url.ParseQuery(decoded)
clientSignature := strings.Replace(params.Get("sig"), " ", "+", -1)
if clientSignature != expectedSignature {
fmt.Printf("Got wrong signature from client.\n- Expected: %s\n- Got: %s\n", expectedSignature, clientSignature)
logger.Errorf("Got wrong signature from client.\n- Expected: %s\n- Got: %s\n", expectedSignature, clientSignature)
c.IndentedJSON(401, gin.H{
"code": "Unauthorized",
"message": "Wrong signature.",

View File

@@ -2,10 +2,10 @@ package middleware
import (
"bytes"
"fmt"
"io"
"github.com/gin-gonic/gin"
"github.com/pikami/cosmium/internal/logger"
)
func RequestLogger() gin.HandlerFunc {
@@ -16,7 +16,7 @@ func RequestLogger() gin.HandlerFunc {
bodyStr := readBody(rdr1)
if bodyStr != "" {
fmt.Println(bodyStr)
logger.Debug(bodyStr)
}
c.Request.Body = rdr2

View File

@@ -26,9 +26,14 @@ func GetPartitionKeyRanges(c *gin.Context) {
c.Header("x-ms-global-committed-lsn", "420")
c.Header("x-ms-item-count", fmt.Sprintf("%d", len(partitionKeyRanges)))
collectionRid := collectionId
collection, _ := repositories.GetCollection(databaseId, collectionId)
if collection.ResourceID != "" {
collectionRid = collection.ResourceID
}
c.IndentedJSON(http.StatusOK, gin.H{
"_rid": collection.ResourceID,
"_rid": collectionRid,
"_count": len(partitionKeyRanges),
"PartitionKeyRanges": partitionKeyRanges,
})

View File

@@ -1,12 +1,42 @@
package handlers
import (
"fmt"
"net/http"
"github.com/gin-gonic/gin"
"github.com/pikami/cosmium/internal/constants"
"github.com/pikami/cosmium/api/config"
)
func GetServerInfo(c *gin.Context) {
c.IndentedJSON(http.StatusOK, constants.ServerInfoResponse)
c.IndentedJSON(http.StatusOK, gin.H{
"_self": "",
"id": config.Config.DatabaseAccount,
"_rid": fmt.Sprintf("%s.%s", config.Config.DatabaseAccount, config.Config.DatabaseDomain),
"media": "//media/",
"addresses": "//addresses/",
"_dbs": "//dbs/",
"writableLocations": []map[string]interface{}{
{
"name": "South Central US",
"databaseAccountEndpoint": config.Config.DatabaseEndpoint,
},
},
"readableLocations": []map[string]interface{}{
{
"name": "South Central US",
"databaseAccountEndpoint": config.Config.DatabaseEndpoint,
},
},
"enableMultipleWriteLocations": false,
"userReplicationPolicy": map[string]interface{}{
"asyncReplication": false,
"minReplicaSetSize": 1,
"maxReplicasetSize": 4,
},
"userConsistencyPolicy": map[string]interface{}{"defaultConsistencyLevel": "Session"},
"systemReplicationPolicy": map[string]interface{}{"minReplicaSetSize": 1, "maxReplicasetSize": 4},
"readPolicy": map[string]interface{}{"primaryReadCoefficient": 1, "secondaryReadCoefficient": 1},
"queryEngineConfiguration": "{\"allowNewKeywords\":true,\"maxJoinsPerSqlQuery\":10,\"maxQueryRequestTimeoutFraction\":0.9,\"maxSqlQueryInputLength\":524288,\"maxUdfRefPerSqlQuery\":10,\"queryMaxInMemorySortDocumentCount\":-1000,\"spatialMaxGeometryPointCount\":256,\"sqlAllowNonFiniteNumbers\":false,\"sqlDisableOptimizationFlags\":0,\"enableSpatialIndexing\":true,\"maxInExpressionItemsCount\":2147483647,\"maxLogicalAndPerSqlQuery\":2147483647,\"maxLogicalOrPerSqlQuery\":2147483647,\"maxSpatialQueryCells\":2147483647,\"sqlAllowAggregateFunctions\":true,\"sqlAllowGroupByClause\":true,\"sqlAllowLike\":true,\"sqlAllowSubQuery\":true,\"sqlAllowScalarSubQuery\":true,\"sqlAllowTop\":true}",
})
}

View File

@@ -1,15 +1,24 @@
package api
import (
"fmt"
"net/http"
"github.com/gin-gonic/gin"
"github.com/pikami/cosmium/api/config"
"github.com/pikami/cosmium/api/handlers"
"github.com/pikami/cosmium/api/handlers/middleware"
"github.com/pikami/cosmium/internal/logger"
tlsprovider "github.com/pikami/cosmium/internal/tls_provider"
)
func CreateRouter() *gin.Engine {
router := gin.Default()
router.Use(middleware.RequestLogger())
if config.Config.Debug {
router.Use(middleware.RequestLogger())
}
router.Use(middleware.Authentication())
router.GET("/dbs/:databaseId/colls/:collId/pkranges", handlers.GetPartitionKeyRanges)
@@ -43,3 +52,43 @@ func CreateRouter() *gin.Engine {
return router
}
func StartAPI() {
if !config.Config.Debug {
gin.SetMode(gin.ReleaseMode)
}
router := CreateRouter()
listenAddress := fmt.Sprintf(":%d", config.Config.Port)
if config.Config.TLS_CertificatePath != "" && config.Config.TLS_CertificateKey != "" {
err := router.RunTLS(
listenAddress,
config.Config.TLS_CertificatePath,
config.Config.TLS_CertificateKey)
if err != nil {
logger.Error("Failed to start HTTPS server:", err)
}
return
}
if config.Config.DisableTls {
router.Run(listenAddress)
}
tlsConfig := tlsprovider.GetDefaultTlsConfig()
server := &http.Server{
Addr: listenAddress,
Handler: router.Handler(),
TLSConfig: tlsConfig,
}
logger.Infof("Listening and serving HTTPS on %s\n", server.Addr)
err := server.ListenAndServeTLS("", "")
if err != nil {
logger.Error("Failed to start HTTPS server:", err)
}
router.Run()
}

125
docs/CODE_OF_CONDUCT.md Normal file
View File

@@ -0,0 +1,125 @@
# Contributor Covenant Code of Conduct
## Our Pledge
We as members, contributors, and leaders pledge to make participation in our
community a harassment-free experience for everyone, regardless of age, body
size, visible or invisible disability, ethnicity, sex characteristics, gender
identity and expression, level of experience, education, socio-economic status,
nationality, personal appearance, race, caste, color, religion, or sexual
identity and orientation.
We pledge to act and interact in ways that contribute to an open, welcoming,
diverse, inclusive, and healthy community.
## Our Standards
Examples of behavior that contributes to a positive environment for our
community include:
* Demonstrating empathy and kindness toward other people
* Being respectful of differing opinions, viewpoints, and experiences
* Giving and gracefully accepting constructive feedback
* Accepting responsibility and apologizing to those affected by our mistakes,
and learning from the experience
* Focusing on what is best not just for us as individuals, but for the overall
community
Examples of unacceptable behavior include:
* The use of sexualized language or imagery, and sexual attention or advances of
any kind
* Trolling, insulting or derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or email address,
without their explicit permission
* Other conduct which could reasonably be considered inappropriate in a
professional setting
## Enforcement Responsibilities
Community leaders are responsible for clarifying and enforcing our standards of
acceptable behavior and will take appropriate and fair corrective action in
response to any behavior that they deem inappropriate, threatening, offensive,
or harmful.
Community leaders have the right and responsibility to remove, edit, or reject
comments, commits, code, wiki edits, issues, and other contributions that are
not aligned to this Code of Conduct, and will communicate reasons for moderation
decisions when appropriate.
## Scope
This Code of Conduct applies within all community spaces, and also applies when
an individual is officially representing the community in public spaces.
Examples of representing our community include using an official email address,
posting via an official social media account, or acting as an appointed
representative at an online or offline event.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported to the community leaders responsible for enforcement at
cosmium@pikami.org.
All complaints will be reviewed and investigated promptly and fairly.
All community leaders are obligated to respect the privacy and security of the
reporter of any incident.
## Enforcement Guidelines
Community leaders will follow these Community Impact Guidelines in determining
the consequences for any action they deem in violation of this Code of Conduct:
### 1. Correction
**Community Impact**: Use of inappropriate language or other behavior deemed
unprofessional or unwelcome in the community.
**Consequence**: A private, written warning from community leaders, providing
clarity around the nature of the violation and an explanation of why the
behavior was inappropriate. A public apology may be requested.
### 2. Warning
**Community Impact**: A violation through a single incident or series of
actions.
**Consequence**: A warning with consequences for continued behavior. No
interaction with the people involved, including unsolicited interaction with
those enforcing the Code of Conduct, for a specified period of time. This
includes avoiding interactions in community spaces as well as external channels
like social media. Violating these terms may lead to a temporary or permanent
ban.
### 3. Temporary Ban
**Community Impact**: A serious violation of community standards, including
sustained inappropriate behavior.
**Consequence**: A temporary ban from any sort of interaction or public
communication with the community for a specified period of time. No public or
private interaction with the people involved, including unsolicited interaction
with those enforcing the Code of Conduct, is allowed during this period.
Violating these terms may lead to a permanent ban.
### 4. Permanent Ban
**Community Impact**: Demonstrating a pattern of violation of community
standards, including sustained inappropriate behavior, harassment of an
individual, or aggression toward or disparagement of classes of individuals.
**Consequence**: A permanent ban from any sort of public interaction within the
community.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage],
version 2.1, available at
[https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1].
Community Impact Guidelines were inspired by
[Mozilla's code of conduct enforcement ladder][Mozilla CoC].
For answers to common questions about this code of conduct, see the FAQ at
[https://www.contributor-covenant.org/faq][FAQ].

203
docs/COMPATIBILITY.md Normal file
View File

@@ -0,0 +1,203 @@
# Cosmium Compatibility with Cosmos DB
## Introduction
Cosmium is designed to emulate the functionality of Cosmos DB, providing developers with a local development environment that closely mimics the behavior of Cosmos DB. While Cosmium aims to be compatible with Cosmos DB, there are certain differences and limitations to be aware of. This document provides an overview of Cosmium's compatibility with Cosmos DB and highlights areas where deviations may occur.
## Supported Features
Cosmium strives to support the core features of Cosmos DB, including:
- REST API
- SQL-like query language
- Document-based data model
## Compatibility Matrix
### Features
| Feature | Implemented |
|-------------------------------|-------------|
| Subqueries | No |
| Joins | No |
| Computed properties | No |
| Coalesce operators | No |
| Bitwise operators | No |
| GeoJSON location data | No |
| Parameterized queries | Yes |
| Stored procedures | No |
| Triggers | No |
| User-defined functions (UDFs) | No |
### Clauses
| Clause | Implemented |
|--------------|-------------|
| SELECT | Yes |
| FROM | Yes |
| WHERE | Yes |
| ORDER BY | Yes |
| GROUP BY | Yes |
| OFFSET LIMIT | Yes |
### Keywords
| Keyword | Implemented |
|----------|-------------|
| BETWEEN | No |
| DISTINCT | Yes |
| LIKE | No |
| IN | Yes |
| TOP | Yes |
### Aggregate Functions
| Function | Implemented |
|----------|-------------|
| AVG | Yes |
| COUNT | Yes |
| MAX | Yes |
| MIN | Yes |
| SUM | Yes |
### Array Functions
| Function | Implemented |
|----------------|-------------|
| ARRAY_CONCAT | Yes |
| ARRAY_CONTAINS | No |
| ARRAY_LENGTH | Yes |
| ARRAY_SLICE | Yes |
| CHOOSE | No |
| ObjectToArray | No |
| SetIntersect | Yes |
| SetUnion | Yes |
### Conditional Functions
| Function | Implemented |
|----------|-------------|
| IIF | No |
### Date and time Functions
| Function | Implemented |
|---------------------------|-------------|
| DateTimeAdd | No |
| DateTimeBin | No |
| DateTimeDiff | No |
| DateTimeFromParts | No |
| DateTimePart | No |
| DateTimeToTicks | No |
| DateTimeToTimestamp | No |
| GetCurrentDateTime | No |
| GetCurrentDateTimeStatic | No |
| GetCurrentTicks | No |
| GetCurrentTicksStatic | No |
| GetCurrentTimestamp | No |
| GetCurrentTimestampStatic | No |
| TicksToDateTime | No |
| TimestampToDateTime | No |
### Item Functions
| Function | Implemented |
|------------|-------------|
| DocumentId | No |
### Mathematical Functions
| Function | Implemented |
|------------------|-------------|
| ABS | No |
| ACOS | No |
| ASIN | No |
| ATAN | No |
| ATN2 | No |
| CEILING | No |
| COS | No |
| COT | No |
| DEGREES | No |
| EXP | No |
| FLOOR | No |
| IntAdd | No |
| IntBitAnd | No |
| IntBitLeftShift | No |
| IntBitNot | No |
| IntBitOr | No |
| IntBitRightShift | No |
| IntBitXor | No |
| IntDiv | No |
| IntMod | No |
| IntMul | No |
| IntSub | No |
| LOG | No |
| LOG10 | No |
| NumberBin | No |
| PI | No |
| POWER | No |
| RADIANS | No |
| RAND | No |
| ROUND | No |
| SIGN | No |
| SIN | No |
| SQRT | No |
| SQUARE | No |
| TAN | No |
| TRUNC | No |
### Spatial Functions
| Function | Implemented |
|--------------------|-------------|
| ST_AREA | No |
| ST_DISTANCE | No |
| ST_WITHIN | No |
| ST_INTERSECTS | No |
| ST_ISVALID | No |
| ST_ISVALIDDETAILED | No |
### String Functions
| Function | Implemented |
|-----------------|-------------|
| CONCAT | Yes |
| CONTAINS | Yes |
| ENDSWITH | Yes |
| INDEX_OF | Yes |
| LEFT | Yes |
| LENGTH | Yes |
| LOWER | Yes |
| LTRIM | Yes |
| REGEXMATCH | No |
| REPLACE | Yes |
| REPLICATE | Yes |
| REVERSE | Yes |
| RIGHT | Yes |
| RTRIM | Yes |
| STARTSWITH | Yes |
| STRINGEQUALS | Yes |
| StringToArray | No |
| StringToBoolean | No |
| StringToNull | No |
| StringToNumber | No |
| StringToObject | No |
| SUBSTRING | Yes |
| ToString | Yes |
| TRIM | Yes |
| UPPER | Yes |
### Type checking Functions
| Function | Implemented |
|------------------|-------------|
| IS_ARRAY | Yes |
| IS_BOOL | Yes |
| IS_DEFINED | Yes |
| IS_FINITE_NUMBER | Yes |
| IS_INTEGER | Yes |
| IS_NULL | Yes |
| IS_NUMBER | Yes |
| IS_OBJECT | Yes |
| IS_PRIMITIVE | Yes |
| IS_STRING | Yes |
## Known Differences
While Cosmium aims to replicate the behavior of Cosmos DB as closely as possible, there are certain differences and limitations to be aware of:
1. **Performance**: Cosmium may exhibit different performance characteristics compared to Cosmos DB, especially under heavy load or large datasets.
2. **Consistency Levels**: The consistency model in Cosmium may differ slightly from Cosmos DB.
3. **Features**: Some advanced features or functionalities of Cosmos DB may not be fully supported or available in Cosmium.
## Future Development
Cosmium is actively developed and maintained, with ongoing efforts to improve compatibility with Cosmos DB and enhance its features and capabilities. Future updates may address known differences and limitations, as well as introduce new functionality to bring Cosmium closer to feature parity with Cosmos DB.

37
docs/CONTRIBUTING.md Normal file
View File

@@ -0,0 +1,37 @@
# Contributing to Cosmium
Thank you for considering contributing to Cosmium! We appreciate your interest in helping to improve our project.
Please note that by participating in this project, you agree to abide by our [Code of Conduct](/docs/CODE_OF_CONDUCT.md). We expect all contributors to uphold the principles of respect, inclusivity, and professionalism.
If you have any questions or need assistance with the contribution process, feel free to reach out to us by opening an issue or contacting the maintainers directly.
We look forward to your contributions! 🚀
## Finding ways to contribute
A great way to contribute is to scan the [Compatibility Matrix](/docs/compatibility.md) for unsupported features and improving compatibility with CosmosDB.
A part from that, the [Issues page](https://github.com/pikami/cosmium/issues) might contain issues registered by other users. Fixing reported issues is a great way to contribute.
## How to Contribute
1. **Create an Issue**: Before starting work on a new feature or bug fix, please create an issue or look for existing ones on the [Issues page](https://github.com/pikami/cosmium/issues) to discuss your proposed changes. This allows us to provide feedback and ensure that your contribution aligns with the project goals.
2. **Fork the Repository**: Once you have identified an issue to work on, fork the repository to your own GitHub account.
3. **Create a Branch**: Create a new branch for your changes using a descriptive name that reflects the issue you are addressing.
4. **Commit Changes**: Commit your changes with clear and descriptive commit messages. Reference the issue number in the commit message. **Please write unit tests for your implemented feature!**
5. **Create a Pull Request**: Once your changes are ready, create a pull request from your forked repository to the main repository. Be sure to include a detailed description of your changes and reference the relevant issue.
6. **Review and Collaborate**: Participate in the code review process by addressing any feedback or comments from maintainers. Collaboration and constructive feedback help ensure the quality of contributions.
## Example Commits
To get an idea of how to implement new query functions, you can review the following example commits:
* [Implement IN function](https://github.com/pikami/cosmium/commit/f37c664c1aef39ee820106eaec1a3708ee7a93c8)
* [Implement ToString function](https://github.com/pikami/cosmium/commit/16f41a547956f54481605f0ce035eee978a5e74b)
* [Implement ARRAY_CONCAT, ARRAY_LENGTH, ARRAY_SLICE, SetIntersect, SetUnion functions](https://github.com/pikami/cosmium/commit/1c5e5ce85d70ed91e4b9be9e8f76d59e6eafc1b5)

View File

@@ -1,43 +1,9 @@
package constants
import (
"fmt"
"github.com/gin-gonic/gin"
"github.com/pikami/cosmium/api/config"
)
var ServerInfoResponse = gin.H{
"_self": "",
"id": config.Config.DatabaseAccount,
"_rid": fmt.Sprintf("%s.%s", config.Config.DatabaseAccount, config.Config.DatabaseDomain),
"media": "//media/",
"addresses": "//addresses/",
"_dbs": "//dbs/",
"writableLocations": []map[string]interface{}{
{
"name": "South Central US",
"databaseAccountEndpoint": config.Config.DatabaseEndpoint,
},
},
"readableLocations": []map[string]interface{}{
{
"name": "South Central US",
"databaseAccountEndpoint": config.Config.DatabaseEndpoint,
},
},
"enableMultipleWriteLocations": false,
"userReplicationPolicy": map[string]interface{}{
"asyncReplication": false,
"minReplicaSetSize": 1,
"maxReplicasetSize": 4,
},
"userConsistencyPolicy": map[string]interface{}{"defaultConsistencyLevel": "Session"},
"systemReplicationPolicy": map[string]interface{}{"minReplicaSetSize": 1, "maxReplicasetSize": 4},
"readPolicy": map[string]interface{}{"primaryReadCoefficient": 1, "secondaryReadCoefficient": 1},
"queryEngineConfiguration": "{\"allowNewKeywords\":true,\"maxJoinsPerSqlQuery\":10,\"maxQueryRequestTimeoutFraction\":0.9,\"maxSqlQueryInputLength\":524288,\"maxUdfRefPerSqlQuery\":10,\"queryMaxInMemorySortDocumentCount\":-1000,\"spatialMaxGeometryPointCount\":256,\"sqlAllowNonFiniteNumbers\":false,\"sqlDisableOptimizationFlags\":0,\"enableSpatialIndexing\":true,\"maxInExpressionItemsCount\":2147483647,\"maxLogicalAndPerSqlQuery\":2147483647,\"maxLogicalOrPerSqlQuery\":2147483647,\"maxSpatialQueryCells\":2147483647,\"sqlAllowAggregateFunctions\":true,\"sqlAllowGroupByClause\":true,\"sqlAllowLike\":true,\"sqlAllowSubQuery\":true,\"sqlAllowScalarSubQuery\":true,\"sqlAllowTop\":true}",
}
var QueryPlanResponse = gin.H{
"partitionedQueryExecutionInfoVersion": 2,
"queryInfo": map[string]interface{}{

40
internal/logger/logger.go Normal file
View File

@@ -0,0 +1,40 @@
package logger
import (
"log"
"os"
"github.com/pikami/cosmium/api/config"
)
var DebugLogger = log.New(os.Stdout, "", log.Ldate|log.Ltime|log.Lshortfile)
var InfoLogger = log.New(os.Stdout, "", log.Ldate|log.Ltime)
var ErrorLogger = log.New(os.Stderr, "", log.Ldate|log.Ltime|log.Lshortfile)
func Debug(v ...any) {
if config.Config.Debug {
DebugLogger.Println(v...)
}
}
func Debugf(format string, v ...any) {
if config.Config.Debug {
DebugLogger.Printf(format, v...)
}
}
func Info(v ...any) {
InfoLogger.Println(v...)
}
func Infof(format string, v ...any) {
InfoLogger.Printf(format, v...)
}
func Error(v ...any) {
ErrorLogger.Println(v...)
}
func Errorf(format string, v ...any) {
ErrorLogger.Printf(format, v...)
}

View File

@@ -10,19 +10,21 @@ import (
// I have no idea what this is tbh
func GetPartitionKeyRanges(databaseId string, collectionId string) ([]repositorymodels.PartitionKeyRange, repositorymodels.RepositoryStatus) {
var ok bool
var database repositorymodels.Database
var collection repositorymodels.Collection
if database, ok = storeState.Databases[databaseId]; !ok {
return make([]repositorymodels.PartitionKeyRange, 0), repositorymodels.StatusNotFound
databaseRid := databaseId
collectionRid := collectionId
var timestamp int64 = 0
if database, ok := storeState.Databases[databaseId]; !ok {
databaseRid = database.ResourceID
}
if collection, ok = storeState.Collections[databaseId][collectionId]; !ok {
return make([]repositorymodels.PartitionKeyRange, 0), repositorymodels.StatusNotFound
if collection, ok := storeState.Collections[databaseId][collectionId]; !ok {
collectionRid = collection.ResourceID
timestamp = collection.TimeStamp
}
pkrResourceId := resourceid.NewCombined(database.ResourceID, collection.ResourceID, resourceid.New())
pkrSelf := fmt.Sprintf("dbs/%s/colls/%s/pkranges/%s/", database.ResourceID, collection.ResourceID, pkrResourceId)
pkrResourceId := resourceid.NewCombined(databaseRid, collectionRid, resourceid.New())
pkrSelf := fmt.Sprintf("dbs/%s/colls/%s/pkranges/%s/", databaseRid, collectionRid, pkrResourceId)
etag := fmt.Sprintf("\"%s\"", uuid.New())
return []repositorymodels.PartitionKeyRange{
@@ -37,7 +39,7 @@ func GetPartitionKeyRanges(databaseId string, collectionId string) ([]repository
ThroughputFraction: 1,
Status: "online",
Parents: []interface{}{},
TimeStamp: collection.TimeStamp,
TimeStamp: timestamp,
Lsn: 17,
},
}, repositorymodels.StatusOk

View File

@@ -2,11 +2,12 @@ package repositories
import (
"encoding/json"
"fmt"
"log"
"os"
"reflect"
"github.com/pikami/cosmium/api/config"
"github.com/pikami/cosmium/internal/logger"
repositorymodels "github.com/pikami/cosmium/internal/repository_models"
)
@@ -19,21 +20,45 @@ var storeState = repositorymodels.State{
Documents: make(map[string]map[string]map[string]repositorymodels.Document),
}
func InitializeRepository() {
if config.Config.InitialDataFilePath != "" {
LoadStateFS(config.Config.InitialDataFilePath)
return
}
if config.Config.PersistDataFilePath != "" {
stat, err := os.Stat(config.Config.PersistDataFilePath)
if err != nil {
return
}
if stat.IsDir() {
logger.Error("Argument '-Persist' must be a path to file, not a directory.")
os.Exit(1)
}
LoadStateFS(config.Config.PersistDataFilePath)
return
}
}
func LoadStateFS(filePath string) {
data, err := os.ReadFile(filePath)
if err != nil {
log.Fatalf("Error reading state JSON file: %v", err)
return
}
var state repositorymodels.State
if err := json.Unmarshal(data, &state); err != nil {
log.Fatalf("Error unmarshalling state JSON: %v", err)
return
}
fmt.Println("Loaded state:")
fmt.Printf("Databases: %d\n", getLength(state.Databases))
fmt.Printf("Collections: %d\n", getLength(state.Collections))
fmt.Printf("Documents: %d\n", getLength(state.Documents))
logger.Info("Loaded state:")
logger.Infof("Databases: %d\n", getLength(state.Databases))
logger.Infof("Collections: %d\n", getLength(state.Collections))
logger.Infof("Documents: %d\n", getLength(state.Documents))
storeState = state
@@ -43,16 +68,16 @@ func LoadStateFS(filePath string) {
func SaveStateFS(filePath string) {
data, err := json.MarshalIndent(storeState, "", "\t")
if err != nil {
fmt.Printf("Failed to save state: %v\n", err)
logger.Errorf("Failed to save state: %v\n", err)
return
}
os.WriteFile(filePath, data, os.ModePerm)
fmt.Println("Saved state:")
fmt.Printf("Databases: %d\n", getLength(storeState.Databases))
fmt.Printf("Collections: %d\n", getLength(storeState.Collections))
fmt.Printf("Documents: %d\n", getLength(storeState.Documents))
logger.Info("Saved state:")
logger.Infof("Databases: %d\n", getLength(storeState.Databases))
logger.Infof("Collections: %d\n", getLength(storeState.Collections))
logger.Infof("Documents: %d\n", getLength(storeState.Documents))
}
func GetState() repositorymodels.State {

View File

@@ -0,0 +1,61 @@
package tlsprovider
const certificate = `
-----BEGIN CERTIFICATE-----
MIIEaDCCAlCgAwIBAgIUAY7ito1IQfbIi52C0evhqHWgEvQwDQYJKoZIhvcNAQEL
BQAwMzELMAkGA1UEBhMCTFQxEjAQBgNVBAgMCUxpdGh1YW5pYTEQMA4GA1UECgwH
Q29zbWl1bTAeFw0yNDAyMjcxOTE4NThaFw0zNDAyMjYxOTE4NThaMD8xCzAJBgNV
BAYTAkxUMRIwEAYDVQQIDAlMaXRodWFuaWExEDAOBgNVBAoMB0Nvc21pdW0xCjAI
BgNVBAMMASowggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCZxGz5clcf
fvE6wS9Q2xPsUjeKdwotRCfKRu9kT7o1cZOSRBp7DgdeLvZ7BzqU1tk5wiLLiZwB
gI6amQAd6z6EwUcUH0mHtFiWU0y/FROz0QUojbbYp0PMUhWjlPAxAGaiwgF/82z7
/lmgMjf5v32XsMfa4U+FaaNYs7gu7aCQBQTAHmOIPnEAeFk9xQ2VzntRUWwzDYOV
SimtPZk2O2X18V8KTgTLMQF1KErIyznIwEPB/BLi+ihLkh/8BaaxoIeOPIhRLNFr
ecZrc/8+S4dUSUQDfmV3JFYFFheG0XIPEwXIaXiDAphpkCGhMIC2pDL8r14sntvn
juHFZxmSP4V5AgMBAAGjaDBmMB8GA1UdIwQYMBaAFEbQ/7hV4FWrptdOk540R2lF
SB1BMAkGA1UdEwQCMAAwCwYDVR0PBAQDAgTwMAwGA1UdEQQFMAOCASowHQYDVR0O
BBYEFGv5XvoFFzrG54GQ+WMFm6UO36BJMA0GCSqGSIb3DQEBCwUAA4ICAQBZh/vZ
PBamebTEpiQz6cgf8+GcTi++ebYUGQ3YJj82pqVBdipOhYQOZJ0fOlT1qRGNglut
+m5zn0iuXsNucP/32xdf1aJBnsU/aGrlf5ohJpGNxYfNPsewxeqQI23Yj22ec1gy
WL2pFDYNyTZMM7Wgys7m3i9lb6TYOF2lNO3WbNuuuETsDAPa0rD0R8QsQOfYOSNJ
YuWE4qZu+ySvTWsMZwlcqs7QL3Sd91UjItIS/AgqbnLvgt4z5ckGCIvickUfAZuQ
6x592hTz4OZ+WIYDejtb5MMXRaKEXgfF6o1idrD7YgVutm+2+mYpN1v9aLbCs7QW
9RkJoTXFQRNGq6j/cO0ZrCKFkttduziMWRz5X9QWADME1NsL53DfDkaxp9Nh+CCu
0S9OF9nVLJVigdXe4O1cQ0Qh633O6k+F/xWYcmMyVt3V2bs7FPfygGUx60tfIbpi
cBK3BsuzUrId3ozvYPsmfxYlzmyspyS6G+f7zLFOakm3fuqDJpnFNXmRY2Ljd3Cp
punuMT6zSctHAxpgJm1g9R6PcaGr+b/n6zkbxyK9+SFzwN3Lb18WFj5OcslNM/g5
ERE5Ws+Vae6MleSmsxSytgH4qn0ormPWuouBLaW0Rv2ZHdkt3myq8kTqtqdw3LRR
ogcLQ3cL6I5FKGjm2TOF72DQHvOol8ck0uMz/w==
-----END CERTIFICATE-----
`
const certificateKey = `
-----BEGIN PRIVATE KEY-----
MIIEuwIBADANBgkqhkiG9w0BAQEFAASCBKUwggShAgEAAoIBAQCZxGz5clcffvE6
wS9Q2xPsUjeKdwotRCfKRu9kT7o1cZOSRBp7DgdeLvZ7BzqU1tk5wiLLiZwBgI6a
mQAd6z6EwUcUH0mHtFiWU0y/FROz0QUojbbYp0PMUhWjlPAxAGaiwgF/82z7/lmg
Mjf5v32XsMfa4U+FaaNYs7gu7aCQBQTAHmOIPnEAeFk9xQ2VzntRUWwzDYOVSimt
PZk2O2X18V8KTgTLMQF1KErIyznIwEPB/BLi+ihLkh/8BaaxoIeOPIhRLNFrecZr
c/8+S4dUSUQDfmV3JFYFFheG0XIPEwXIaXiDAphpkCGhMIC2pDL8r14sntvnjuHF
ZxmSP4V5AgMBAAECgf89wcgjpZnzoWoiM3Z6QDJnkiUdXQumHQracBnRFXnMy8p9
wCd4ecnu9ptd8OArXgVMiaILWZeGXlqtW872m6Lej6DrJkpOt3NG9CvscdaHdthW
9dzv8d7IEtuRN4/WWOm7Tke7eD7763ta9i9/niR2q7DazPVw8vYhkyoNe864qVrq
Vw6+MMetz3TDHZ68p17yJJ9FJ0z0vHj3KJFrxnJonMe+/LcQX490y4zZw+zeyCkh
y/bsgvFGhnUhJ+mOz+qv0KL7HyUR69p9/+mjQH+AQH+j24xgd1IL0Dror9Cy1kxY
uKmi8pN1y288GmjkWosGMb0p3Pse1OkOyYFIbxECgYEA2ED3PSPoHWLHfKhg2BFw
yMPtern06rjKuwMNlD+mKS66Z+OsQi2EBsqomGnr1HGvYgQik0jwMcx0+Sup9/Zp
az8ebH6S4Tdxmnlwn34lhTIAF1KJS19AYvbhOydV+M+hq7Y7QxTqYsJAgEYwsozQ
0XeAzRBIiRxdcMFHP40zZIkCgYEAtgdiwo5d5iyvXEqx/5+NdM4b/ImrbaFIAb0v
MqiPpOA/+7EKlx72gJKVKh2iv4jvEUfduNEUXt77Yqo66HhfiTBVYxYwThK8E0Mq
TSKKdJsdPSThLS3qjeARpzQpWLiBZH90GxbfFL3ogIOa/UcgwRrqPc5a/yq8adSs
KGrfvXECgYEAmSMAMbqgn1aY32y5D6jiDjm4jMTsa98qKN5TmlysRNODSxhNnptu
uASA+VVgnBNZV/aHqXboKMuZNe22shI7uqd62ueTCYtiljpTB46j8TtkFx/qe4Zb
KPmcq3ACkGwwF1G3i5xfEkputKd/yqCvKvYOLqjORNHiVXt5Acby0skCgYBYkZ9s
KvllVbi9n1qclnWtr9vONO5EmYT/051zeLDr+HEpditA/L/UL36Ez4awy2AHeIBZ
vOG8h6Kpj0q6cleJ2Qqy+8jlNBhvBu8+OOBFfHPtnFQ0N3M5NR1hze+QS7YpwBou
VCKXZRAL9/0h38oAK6huCkocfh7PH7vkrpvPAQKBgCFDDtk7aBJsNcOW+aq4IEvf
nZ5hhhdelNLeN29RrJ71GwJrCG3NbhopWlCDqZ/Dd6QoEUpebqvlMGvQJBuz/QKb
ilcZlmaCS9pqIXAFK9GQ89V/xa8OibOuJUiBgShnfSQqAwQrfX1vYjtKErnjoRFs
9+zaWugLCC47Hw6QlMDa
-----END PRIVATE KEY-----
`

View File

@@ -0,0 +1,19 @@
package tlsprovider
import (
"crypto/tls"
"github.com/pikami/cosmium/internal/logger"
)
func GetDefaultTlsConfig() *tls.Config {
cert, err := tls.X509KeyPair([]byte(certificate), []byte(certificateKey))
if err != nil {
logger.Error("Failed to parse certificate and key:", err)
return &tls.Config{}
}
return &tls.Config{
Certificates: []tls.Certificate{cert},
}
}

16
main.go
View File

@@ -1,7 +1,6 @@
package main
import (
"fmt"
"os"
"os/signal"
"syscall"
@@ -14,20 +13,9 @@ import (
func main() {
config.ParseFlags()
if config.Config.InitialDataFilePath != "" {
repositories.LoadStateFS(config.Config.InitialDataFilePath)
}
repositories.InitializeRepository()
router := api.CreateRouter()
if config.Config.TLS_CertificatePath == "" ||
config.Config.TLS_CertificateKey == "" {
go router.Run(fmt.Sprintf(":%d", config.Config.Port))
} else {
go router.RunTLS(
fmt.Sprintf(":%d", config.Config.Port),
config.Config.TLS_CertificatePath,
config.Config.TLS_CertificateKey)
}
go api.StartAPI()
waitForExit()
}

View File

@@ -4,9 +4,12 @@ type SelectStmt struct {
SelectItems []SelectItem
Table Table
Filters interface{}
Distinct bool
Count int
Offset int
Parameters map[string]interface{}
OrderExpressions []OrderExpression
GroupBy []SelectItem
}
type Table struct {
@@ -117,9 +120,23 @@ const (
FunctionCallSetIntersect FunctionCallType = "SetIntersect"
FunctionCallSetUnion FunctionCallType = "SetUnion"
FunctionCallAggregateAvg FunctionCallType = "AggregateAvg"
FunctionCallAggregateCount FunctionCallType = "AggregateCount"
FunctionCallAggregateMax FunctionCallType = "AggregateMax"
FunctionCallAggregateMin FunctionCallType = "AggregateMin"
FunctionCallAggregateSum FunctionCallType = "AggregateSum"
FunctionCallIn FunctionCallType = "In"
)
var AggregateFunctions = []FunctionCallType{
FunctionCallAggregateAvg,
FunctionCallAggregateCount,
FunctionCallAggregateMax,
FunctionCallAggregateMin,
FunctionCallAggregateSum,
}
type FunctionCall struct {
Arguments []interface{}
Type FunctionCallType

View File

@@ -0,0 +1,130 @@
package nosql_test
import (
"testing"
"github.com/pikami/cosmium/parsers"
)
func Test_Parse_AggregateFunctions(t *testing.T) {
t.Run("Should parse function AVG()", func(t *testing.T) {
testQueryParse(
t,
`SELECT AVG(c.a1) FROM c`,
parsers.SelectStmt{
SelectItems: []parsers.SelectItem{
{
Type: parsers.SelectItemTypeFunctionCall,
Value: parsers.FunctionCall{
Type: parsers.FunctionCallAggregateAvg,
Arguments: []interface{}{
parsers.SelectItem{
Path: []string{"c", "a1"},
Type: parsers.SelectItemTypeField,
},
},
},
},
},
Table: parsers.Table{Value: "c"},
},
)
})
t.Run("Should parse function COUNT()", func(t *testing.T) {
testQueryParse(
t,
`SELECT COUNT(c.a1) FROM c`,
parsers.SelectStmt{
SelectItems: []parsers.SelectItem{
{
Type: parsers.SelectItemTypeFunctionCall,
Value: parsers.FunctionCall{
Type: parsers.FunctionCallAggregateCount,
Arguments: []interface{}{
parsers.SelectItem{
Path: []string{"c", "a1"},
Type: parsers.SelectItemTypeField,
},
},
},
},
},
Table: parsers.Table{Value: "c"},
},
)
})
t.Run("Should parse function MAX()", func(t *testing.T) {
testQueryParse(
t,
`SELECT MAX(c.a1) FROM c`,
parsers.SelectStmt{
SelectItems: []parsers.SelectItem{
{
Type: parsers.SelectItemTypeFunctionCall,
Value: parsers.FunctionCall{
Type: parsers.FunctionCallAggregateMax,
Arguments: []interface{}{
parsers.SelectItem{
Path: []string{"c", "a1"},
Type: parsers.SelectItemTypeField,
},
},
},
},
},
Table: parsers.Table{Value: "c"},
},
)
})
t.Run("Should parse function MIN()", func(t *testing.T) {
testQueryParse(
t,
`SELECT MIN(c.a1) FROM c`,
parsers.SelectStmt{
SelectItems: []parsers.SelectItem{
{
Type: parsers.SelectItemTypeFunctionCall,
Value: parsers.FunctionCall{
Type: parsers.FunctionCallAggregateMin,
Arguments: []interface{}{
parsers.SelectItem{
Path: []string{"c", "a1"},
Type: parsers.SelectItemTypeField,
},
},
},
},
},
Table: parsers.Table{Value: "c"},
},
)
})
t.Run("Should parse function SUM()", func(t *testing.T) {
testQueryParse(
t,
`SELECT SUM(c.a1) FROM c`,
parsers.SelectStmt{
SelectItems: []parsers.SelectItem{
{
Type: parsers.SelectItemTypeFunctionCall,
Value: parsers.FunctionCall{
Type: parsers.FunctionCallAggregateSum,
Arguments: []interface{}{
parsers.SelectItem{
Path: []string{"c", "a1"},
Type: parsers.SelectItemTypeField,
},
},
},
},
},
Table: parsers.Table{Value: "c"},
},
)
})
}

View File

@@ -63,6 +63,24 @@ func Test_Parse(t *testing.T) {
)
})
t.Run("Should parse SELECT with GROUP BY", func(t *testing.T) {
testQueryParse(
t,
`SELECT c.id, c["pk"] FROM c GROUP BY c.id, c.pk`,
parsers.SelectStmt{
SelectItems: []parsers.SelectItem{
{Path: []string{"c", "id"}},
{Path: []string{"c", "pk"}},
},
Table: parsers.Table{Value: "c"},
GroupBy: []parsers.SelectItem{
{Path: []string{"c", "id"}},
{Path: []string{"c", "pk"}},
},
},
)
})
t.Run("Should parse IN function", func(t *testing.T) {
testQueryParse(
t,

File diff suppressed because it is too large Load Diff

View File

@@ -3,24 +3,47 @@ package nosql
import "github.com/pikami/cosmium/parsers"
func makeSelectStmt(columns, table, whereClause interface{}, count interface{}, orderList interface{}) (parsers.SelectStmt, error) {
func makeSelectStmt(
columns, table,
whereClause interface{}, distinctClause interface{},
count interface{}, groupByClause interface{}, orderList interface{},
offsetClause interface{},
) (parsers.SelectStmt, error) {
selectStmt := parsers.SelectStmt{
SelectItems: columns.([]parsers.SelectItem),
Table: table.(parsers.Table),
}
switch v := whereClause.(type) {
case parsers.ComparisonExpression, parsers.LogicalExpression, parsers.Constant, parsers.SelectItem:
selectStmt.Filters = v
}
switch v := whereClause.(type) {
case parsers.ComparisonExpression, parsers.LogicalExpression, parsers.Constant, parsers.SelectItem:
selectStmt.Filters = v
}
if distinctClause != nil {
selectStmt.Distinct = true
}
if n, ok := count.(int); ok {
selectStmt.Count = n
}
if orderExpressions, ok := orderList.([]parsers.OrderExpression); ok {
selectStmt.OrderExpressions = orderExpressions
}
if offsetArr, ok := offsetClause.([]interface{}); ok && len(offsetArr) == 2 {
if n, ok := offsetArr[0].(int); ok {
selectStmt.Offset = n
}
if n, ok := offsetArr[1].(int); ok {
selectStmt.Count = n
}
}
if orderExpressions, ok := orderList.([]parsers.OrderExpression); ok {
selectStmt.OrderExpressions = orderExpressions
}
if groupByClause != nil {
selectStmt.GroupBy = groupByClause.([]parsers.SelectItem)
}
return selectStmt, nil
}
@@ -136,17 +159,28 @@ Input <- selectStmt:SelectStmt {
return selectStmt, nil
}
SelectStmt <- Select ws topClause:TopClause? ws columns:Selection ws
SelectStmt <- Select ws
distinctClause:DistinctClause? ws
topClause:TopClause? ws columns:Selection ws
From ws table:TableName ws
whereClause:(ws Where ws condition:Condition { return condition, nil })?
orderByClause:OrderByClause? {
return makeSelectStmt(columns, table, whereClause, topClause, orderByClause)
groupByClause:(ws GroupBy ws columns:ColumnList { return columns, nil })?
orderByClause:OrderByClause?
offsetClause:OffsetClause? {
return makeSelectStmt(columns, table, whereClause,
distinctClause, topClause, groupByClause, orderByClause, offsetClause)
}
DistinctClause <- "DISTINCT"i
TopClause <- Top ws count:Integer {
return count, nil
}
OffsetClause <- "OFFSET"i ws offset:IntegerLiteral ws "LIMIT"i ws limit:IntegerLiteral {
return []interface{}{offset.(parsers.Constant).Value, limit.(parsers.Constant).Value}, nil
}
Selection <- SelectValueSpec / ColumnList / SelectAsterisk
SelectAsterisk <- "*" {
@@ -273,6 +307,8 @@ And <- "AND"i
Or <- "OR"i
GroupBy <- "GROUP"i ws "BY"i
OrderBy <- "ORDER"i ws "BY"i
ComparisonOperator <- ("=" / "!=" / "<" / "<=" / ">" / ">=") {
@@ -307,6 +343,7 @@ FunctionCall <- StringFunctions
/ TypeCheckingFunctions
/ ArrayFunctions
/ InFunction
/ AggregateFunctions
StringFunctions <- StringEqualsExpression
/ ToStringExpression
@@ -336,6 +373,12 @@ TypeCheckingFunctions <- IsDefined
/ IsPrimitive
/ IsString
AggregateFunctions <- AvgAggregateExpression
/ CountAggregateExpression
/ MaxAggregateExpression
/ MinAggregateExpression
/ SumAggregateExpression
ArrayFunctions <- ArrayConcatExpression
/ ArrayLengthExpression
/ ArraySliceExpression
@@ -489,6 +532,26 @@ InFunction <- ex1:SelectProperty ws "IN"i ws "(" ws ex2:SelectItem others:(ws ",
return createFunctionCall(parsers.FunctionCallIn, append([]interface{}{ex1, ex2}, others.([]interface{})...))
}
AvgAggregateExpression <- "AVG"i "(" ws ex:SelectItem ws ")" {
return createFunctionCall(parsers.FunctionCallAggregateAvg, []interface{}{ex})
}
CountAggregateExpression <- "COUNT"i "(" ws ex:SelectItem ws ")" {
return createFunctionCall(parsers.FunctionCallAggregateCount, []interface{}{ex})
}
MaxAggregateExpression <- "MAX"i "(" ws ex:SelectItem ws ")" {
return createFunctionCall(parsers.FunctionCallAggregateMax, []interface{}{ex})
}
MinAggregateExpression <- "MIN"i "(" ws ex:SelectItem ws ")" {
return createFunctionCall(parsers.FunctionCallAggregateMin, []interface{}{ex})
}
SumAggregateExpression <- "SUM"i "(" ws ex:SelectItem ws ")" {
return createFunctionCall(parsers.FunctionCallAggregateSum, []interface{}{ex})
}
Integer <- [0-9]+ {
return strconv.Atoi(string(c.text))
}

View File

@@ -22,6 +22,20 @@ func Test_Parse_Select(t *testing.T) {
)
})
t.Run("Should parse SELECT DISTINCT", func(t *testing.T) {
testQueryParse(
t,
`SELECT DISTINCT c.id FROM c`,
parsers.SelectStmt{
SelectItems: []parsers.SelectItem{
{Path: []string{"c", "id"}},
},
Table: parsers.Table{Value: "c"},
Distinct: true,
},
)
})
t.Run("Should parse SELECT TOP", func(t *testing.T) {
testQueryParse(
t,
@@ -36,6 +50,21 @@ func Test_Parse_Select(t *testing.T) {
)
})
t.Run("Should parse SELECT OFFSET", func(t *testing.T) {
testQueryParse(
t,
`SELECT c.id FROM c OFFSET 3 LIMIT 5`,
parsers.SelectStmt{
SelectItems: []parsers.SelectItem{
{Path: []string{"c", "id"}},
},
Table: parsers.Table{Value: "c"},
Count: 5,
Offset: 3,
},
)
})
t.Run("Should parse SELECT VALUE", func(t *testing.T) {
testQueryParse(
t,

View File

@@ -0,0 +1,131 @@
package memoryexecutor
import (
"math"
"github.com/pikami/cosmium/parsers"
)
func (c memoryExecutorContext) aggregate_Avg(arguments []interface{}, row RowType) interface{} {
selectExpression := arguments[0].(parsers.SelectItem)
sum := 0.0
count := 0
if array, isArray := row.([]RowType); isArray {
for _, item := range array {
value := c.getFieldValue(selectExpression, item)
if numericValue, ok := value.(float64); ok {
sum += numericValue
count++
} else if numericValue, ok := value.(int); ok {
sum += float64(numericValue)
count++
}
}
}
if count > 0 {
return sum / float64(count)
} else {
return nil
}
}
func (c memoryExecutorContext) aggregate_Count(arguments []interface{}, row RowType) interface{} {
selectExpression := arguments[0].(parsers.SelectItem)
count := 0
if array, isArray := row.([]RowType); isArray {
for _, item := range array {
value := c.getFieldValue(selectExpression, item)
if value != nil {
count++
}
}
}
return count
}
func (c memoryExecutorContext) aggregate_Max(arguments []interface{}, row RowType) interface{} {
selectExpression := arguments[0].(parsers.SelectItem)
max := 0.0
count := 0
if array, isArray := row.([]RowType); isArray {
for _, item := range array {
value := c.getFieldValue(selectExpression, item)
if numericValue, ok := value.(float64); ok {
if numericValue > max {
max = numericValue
}
count++
} else if numericValue, ok := value.(int); ok {
if float64(numericValue) > max {
max = float64(numericValue)
}
count++
}
}
}
if count > 0 {
return max
} else {
return nil
}
}
func (c memoryExecutorContext) aggregate_Min(arguments []interface{}, row RowType) interface{} {
selectExpression := arguments[0].(parsers.SelectItem)
min := math.MaxFloat64
count := 0
if array, isArray := row.([]RowType); isArray {
for _, item := range array {
value := c.getFieldValue(selectExpression, item)
if numericValue, ok := value.(float64); ok {
if numericValue < min {
min = numericValue
}
count++
} else if numericValue, ok := value.(int); ok {
if float64(numericValue) < min {
min = float64(numericValue)
}
count++
}
}
}
if count > 0 {
return min
} else {
return nil
}
}
func (c memoryExecutorContext) aggregate_Sum(arguments []interface{}, row RowType) interface{} {
selectExpression := arguments[0].(parsers.SelectItem)
sum := 0.0
count := 0
if array, isArray := row.([]RowType); isArray {
for _, item := range array {
value := c.getFieldValue(selectExpression, item)
if numericValue, ok := value.(float64); ok {
sum += numericValue
count++
} else if numericValue, ok := value.(int); ok {
sum += float64(numericValue)
count++
}
}
}
if count > 0 {
return sum
} else {
return nil
}
}

View File

@@ -0,0 +1,210 @@
package memoryexecutor_test
import (
"testing"
"github.com/pikami/cosmium/parsers"
memoryexecutor "github.com/pikami/cosmium/query_executors/memory_executor"
)
func Test_Execute_AggregateFunctions(t *testing.T) {
mockData := []memoryexecutor.RowType{
map[string]interface{}{"id": "123", "number": 123, "key": "a"},
map[string]interface{}{"id": "456", "number": 456, "key": "a"},
map[string]interface{}{"id": "789", "number": 789, "key": "b"},
map[string]interface{}{"id": "no-number", "key": "b"},
}
t.Run("Should execute function AVG()", func(t *testing.T) {
testQueryExecute(
t,
parsers.SelectStmt{
SelectItems: []parsers.SelectItem{
{Path: []string{"c", "key"}},
{
Alias: "avg",
Type: parsers.SelectItemTypeFunctionCall,
Value: parsers.FunctionCall{
Type: parsers.FunctionCallAggregateAvg,
Arguments: []interface{}{
parsers.SelectItem{
Path: []string{"c", "number"},
Type: parsers.SelectItemTypeField,
},
},
},
},
},
GroupBy: []parsers.SelectItem{
{Path: []string{"c", "key"}},
},
Table: parsers.Table{Value: "c"},
},
mockData,
[]memoryexecutor.RowType{
map[string]interface{}{"key": "a", "avg": 289.5},
map[string]interface{}{"key": "b", "avg": 789.0},
},
)
})
t.Run("Should execute function AVG() without GROUP BY clause", func(t *testing.T) {
testQueryExecute(
t,
parsers.SelectStmt{
SelectItems: []parsers.SelectItem{
{
Alias: "avg",
Type: parsers.SelectItemTypeFunctionCall,
Value: parsers.FunctionCall{
Type: parsers.FunctionCallAggregateAvg,
Arguments: []interface{}{
parsers.SelectItem{
Path: []string{"c", "number"},
Type: parsers.SelectItemTypeField,
},
},
},
},
},
Table: parsers.Table{Value: "c"},
},
mockData,
[]memoryexecutor.RowType{
map[string]interface{}{"avg": 456.0},
},
)
})
t.Run("Should execute function COUNT()", func(t *testing.T) {
testQueryExecute(
t,
parsers.SelectStmt{
SelectItems: []parsers.SelectItem{
{Path: []string{"c", "key"}},
{
Alias: "cnt",
Type: parsers.SelectItemTypeFunctionCall,
Value: parsers.FunctionCall{
Type: parsers.FunctionCallAggregateCount,
Arguments: []interface{}{
parsers.SelectItem{
Path: []string{"c", "number"},
Type: parsers.SelectItemTypeField,
},
},
},
},
},
GroupBy: []parsers.SelectItem{
{Path: []string{"c", "key"}},
},
Table: parsers.Table{Value: "c"},
},
mockData,
[]memoryexecutor.RowType{
map[string]interface{}{"key": "a", "cnt": 2},
map[string]interface{}{"key": "b", "cnt": 1},
},
)
})
t.Run("Should execute function MAX()", func(t *testing.T) {
testQueryExecute(
t,
parsers.SelectStmt{
SelectItems: []parsers.SelectItem{
{Path: []string{"c", "key"}},
{
Alias: "max",
Type: parsers.SelectItemTypeFunctionCall,
Value: parsers.FunctionCall{
Type: parsers.FunctionCallAggregateMax,
Arguments: []interface{}{
parsers.SelectItem{
Path: []string{"c", "number"},
Type: parsers.SelectItemTypeField,
},
},
},
},
},
GroupBy: []parsers.SelectItem{
{Path: []string{"c", "key"}},
},
Table: parsers.Table{Value: "c"},
},
mockData,
[]memoryexecutor.RowType{
map[string]interface{}{"key": "a", "max": 456.0},
map[string]interface{}{"key": "b", "max": 789.0},
},
)
})
t.Run("Should execute function MIN()", func(t *testing.T) {
testQueryExecute(
t,
parsers.SelectStmt{
SelectItems: []parsers.SelectItem{
{Path: []string{"c", "key"}},
{
Alias: "min",
Type: parsers.SelectItemTypeFunctionCall,
Value: parsers.FunctionCall{
Type: parsers.FunctionCallAggregateMin,
Arguments: []interface{}{
parsers.SelectItem{
Path: []string{"c", "number"},
Type: parsers.SelectItemTypeField,
},
},
},
},
},
GroupBy: []parsers.SelectItem{
{Path: []string{"c", "key"}},
},
Table: parsers.Table{Value: "c"},
},
mockData,
[]memoryexecutor.RowType{
map[string]interface{}{"key": "a", "min": 123.0},
map[string]interface{}{"key": "b", "min": 789.0},
},
)
})
t.Run("Should execute function SUM()", func(t *testing.T) {
testQueryExecute(
t,
parsers.SelectStmt{
SelectItems: []parsers.SelectItem{
{Path: []string{"c", "key"}},
{
Alias: "sum",
Type: parsers.SelectItemTypeFunctionCall,
Value: parsers.FunctionCall{
Type: parsers.FunctionCallAggregateSum,
Arguments: []interface{}{
parsers.SelectItem{
Path: []string{"c", "number"},
Type: parsers.SelectItemTypeField,
},
},
},
},
},
GroupBy: []parsers.SelectItem{
{Path: []string{"c", "key"}},
},
Table: parsers.Table{Value: "c"},
},
mockData,
[]memoryexecutor.RowType{
map[string]interface{}{"key": "a", "sum": 579.0},
map[string]interface{}{"key": "b", "sum": 789.0},
},
)
})
}

View File

@@ -1,9 +1,9 @@
package memoryexecutor
import (
"fmt"
"reflect"
"github.com/pikami/cosmium/internal/logger"
"github.com/pikami/cosmium/parsers"
)
@@ -36,13 +36,13 @@ func (c memoryExecutorContext) array_Slice(arguments []interface{}, row RowType)
lengthEx := c.getFieldValue(arguments[2].(parsers.SelectItem), row)
if length, ok = lengthEx.(int); !ok {
fmt.Println("array_Slice - got length parameters of wrong type")
logger.Error("array_Slice - got length parameters of wrong type")
return []interface{}{}
}
}
if start, ok = startEx.(int); !ok {
fmt.Println("array_Slice - got start parameters of wrong type")
logger.Error("array_Slice - got start parameters of wrong type")
return []interface{}{}
}
@@ -117,7 +117,7 @@ func (c memoryExecutorContext) parseArray(argument interface{}, row RowType) []i
arrValue := reflect.ValueOf(ex)
if arrValue.Kind() != reflect.Slice {
fmt.Println("parseArray got parameters of wrong type")
logger.Error("parseArray got parameters of wrong type")
return nil
}

View File

@@ -6,7 +6,9 @@ import (
"sort"
"strings"
"github.com/pikami/cosmium/internal/logger"
"github.com/pikami/cosmium/parsers"
"golang.org/x/exp/slices"
)
type RowType interface{}
@@ -35,6 +37,33 @@ func Execute(query parsers.SelectStmt, data []RowType) []RowType {
ctx.orderBy(query.OrderExpressions, result)
}
// Apply group
isGroupSelect := query.GroupBy != nil && len(query.GroupBy) > 0
if isGroupSelect {
result = ctx.groupBy(query, result)
}
// Apply select
if !isGroupSelect {
selectedData := make([]RowType, 0)
if hasAggregateFunctions(query.SelectItems) {
// When can have aggregate functions without GROUP BY clause,
// we should aggregate all rows in that case
selectedData = append(selectedData, ctx.selectRow(query.SelectItems, result))
} else {
for _, row := range result {
selectedData = append(selectedData, ctx.selectRow(query.SelectItems, row))
}
}
result = selectedData
}
// Apply distinct
if query.Distinct {
result = deduplicate(result)
}
// Apply result limit
if query.Count > 0 {
count := func() int {
@@ -46,13 +75,7 @@ func Execute(query parsers.SelectStmt, data []RowType) []RowType {
result = result[:count]
}
// Apply select
selectedData := make([]RowType, 0)
for _, row := range result {
selectedData = append(selectedData, ctx.selectRow(query.SelectItems, row))
}
return selectedData
return result
}
func (c memoryExecutorContext) selectRow(selectItems []parsers.SelectItem, row RowType) interface{} {
@@ -162,7 +185,7 @@ func (c memoryExecutorContext) getFieldValue(field parsers.SelectItem, row RowTy
var ok bool
if typedValue, ok = field.Value.(parsers.Constant); !ok {
// TODO: Handle error
fmt.Println("parsers.Constant has incorrect Value type")
logger.Error("parsers.Constant has incorrect Value type")
}
if typedValue.Type == parsers.ConstantTypeParameterConstant &&
@@ -175,92 +198,109 @@ func (c memoryExecutorContext) getFieldValue(field parsers.SelectItem, row RowTy
return typedValue.Value
}
rowValue := row
if array, isArray := row.([]RowType); isArray {
rowValue = array[0]
}
if field.Type == parsers.SelectItemTypeFunctionCall {
var typedValue parsers.FunctionCall
var ok bool
if typedValue, ok = field.Value.(parsers.FunctionCall); !ok {
// TODO: Handle error
fmt.Println("parsers.Constant has incorrect Value type")
logger.Error("parsers.Constant has incorrect Value type")
}
switch typedValue.Type {
case parsers.FunctionCallStringEquals:
return c.strings_StringEquals(typedValue.Arguments, row)
return c.strings_StringEquals(typedValue.Arguments, rowValue)
case parsers.FunctionCallContains:
return c.strings_Contains(typedValue.Arguments, row)
return c.strings_Contains(typedValue.Arguments, rowValue)
case parsers.FunctionCallEndsWith:
return c.strings_EndsWith(typedValue.Arguments, row)
return c.strings_EndsWith(typedValue.Arguments, rowValue)
case parsers.FunctionCallStartsWith:
return c.strings_StartsWith(typedValue.Arguments, row)
return c.strings_StartsWith(typedValue.Arguments, rowValue)
case parsers.FunctionCallConcat:
return c.strings_Concat(typedValue.Arguments, row)
return c.strings_Concat(typedValue.Arguments, rowValue)
case parsers.FunctionCallIndexOf:
return c.strings_IndexOf(typedValue.Arguments, row)
return c.strings_IndexOf(typedValue.Arguments, rowValue)
case parsers.FunctionCallToString:
return c.strings_ToString(typedValue.Arguments, row)
return c.strings_ToString(typedValue.Arguments, rowValue)
case parsers.FunctionCallUpper:
return c.strings_Upper(typedValue.Arguments, row)
return c.strings_Upper(typedValue.Arguments, rowValue)
case parsers.FunctionCallLower:
return c.strings_Lower(typedValue.Arguments, row)
return c.strings_Lower(typedValue.Arguments, rowValue)
case parsers.FunctionCallLeft:
return c.strings_Left(typedValue.Arguments, row)
return c.strings_Left(typedValue.Arguments, rowValue)
case parsers.FunctionCallLength:
return c.strings_Length(typedValue.Arguments, row)
return c.strings_Length(typedValue.Arguments, rowValue)
case parsers.FunctionCallLTrim:
return c.strings_LTrim(typedValue.Arguments, row)
return c.strings_LTrim(typedValue.Arguments, rowValue)
case parsers.FunctionCallReplace:
return c.strings_Replace(typedValue.Arguments, row)
return c.strings_Replace(typedValue.Arguments, rowValue)
case parsers.FunctionCallReplicate:
return c.strings_Replicate(typedValue.Arguments, row)
return c.strings_Replicate(typedValue.Arguments, rowValue)
case parsers.FunctionCallReverse:
return c.strings_Reverse(typedValue.Arguments, row)
return c.strings_Reverse(typedValue.Arguments, rowValue)
case parsers.FunctionCallRight:
return c.strings_Right(typedValue.Arguments, row)
return c.strings_Right(typedValue.Arguments, rowValue)
case parsers.FunctionCallRTrim:
return c.strings_RTrim(typedValue.Arguments, row)
return c.strings_RTrim(typedValue.Arguments, rowValue)
case parsers.FunctionCallSubstring:
return c.strings_Substring(typedValue.Arguments, row)
return c.strings_Substring(typedValue.Arguments, rowValue)
case parsers.FunctionCallTrim:
return c.strings_Trim(typedValue.Arguments, row)
return c.strings_Trim(typedValue.Arguments, rowValue)
case parsers.FunctionCallIsDefined:
return c.typeChecking_IsDefined(typedValue.Arguments, row)
return c.typeChecking_IsDefined(typedValue.Arguments, rowValue)
case parsers.FunctionCallIsArray:
return c.typeChecking_IsArray(typedValue.Arguments, row)
return c.typeChecking_IsArray(typedValue.Arguments, rowValue)
case parsers.FunctionCallIsBool:
return c.typeChecking_IsBool(typedValue.Arguments, row)
return c.typeChecking_IsBool(typedValue.Arguments, rowValue)
case parsers.FunctionCallIsFiniteNumber:
return c.typeChecking_IsFiniteNumber(typedValue.Arguments, row)
return c.typeChecking_IsFiniteNumber(typedValue.Arguments, rowValue)
case parsers.FunctionCallIsInteger:
return c.typeChecking_IsInteger(typedValue.Arguments, row)
return c.typeChecking_IsInteger(typedValue.Arguments, rowValue)
case parsers.FunctionCallIsNull:
return c.typeChecking_IsNull(typedValue.Arguments, row)
return c.typeChecking_IsNull(typedValue.Arguments, rowValue)
case parsers.FunctionCallIsNumber:
return c.typeChecking_IsNumber(typedValue.Arguments, row)
return c.typeChecking_IsNumber(typedValue.Arguments, rowValue)
case parsers.FunctionCallIsObject:
return c.typeChecking_IsObject(typedValue.Arguments, row)
return c.typeChecking_IsObject(typedValue.Arguments, rowValue)
case parsers.FunctionCallIsPrimitive:
return c.typeChecking_IsPrimitive(typedValue.Arguments, row)
return c.typeChecking_IsPrimitive(typedValue.Arguments, rowValue)
case parsers.FunctionCallIsString:
return c.typeChecking_IsString(typedValue.Arguments, row)
return c.typeChecking_IsString(typedValue.Arguments, rowValue)
case parsers.FunctionCallArrayConcat:
return c.array_Concat(typedValue.Arguments, row)
return c.array_Concat(typedValue.Arguments, rowValue)
case parsers.FunctionCallArrayLength:
return c.array_Length(typedValue.Arguments, row)
return c.array_Length(typedValue.Arguments, rowValue)
case parsers.FunctionCallArraySlice:
return c.array_Slice(typedValue.Arguments, row)
return c.array_Slice(typedValue.Arguments, rowValue)
case parsers.FunctionCallSetIntersect:
return c.set_Intersect(typedValue.Arguments, row)
return c.set_Intersect(typedValue.Arguments, rowValue)
case parsers.FunctionCallSetUnion:
return c.set_Union(typedValue.Arguments, row)
return c.set_Union(typedValue.Arguments, rowValue)
case parsers.FunctionCallAggregateAvg:
return c.aggregate_Avg(typedValue.Arguments, row)
case parsers.FunctionCallAggregateCount:
return c.aggregate_Count(typedValue.Arguments, row)
case parsers.FunctionCallAggregateMax:
return c.aggregate_Max(typedValue.Arguments, row)
case parsers.FunctionCallAggregateMin:
return c.aggregate_Min(typedValue.Arguments, row)
case parsers.FunctionCallAggregateSum:
return c.aggregate_Sum(typedValue.Arguments, row)
case parsers.FunctionCallIn:
return c.misc_In(typedValue.Arguments, row)
return c.misc_In(typedValue.Arguments, rowValue)
}
}
value := row
value := rowValue
if len(field.Path) > 1 {
for _, pathSegment := range field.Path[1:] {
if nestedValue, ok := value.(map[string]interface{}); ok {
@@ -282,7 +322,7 @@ func (c memoryExecutorContext) getExpressionParameterValue(
return c.getFieldValue(typedParameter, row)
}
fmt.Println("getExpressionParameterValue - got incorrect parameter type")
logger.Error("getExpressionParameterValue - got incorrect parameter type")
return nil
}
@@ -307,6 +347,47 @@ func (c memoryExecutorContext) orderBy(orderBy []parsers.OrderExpression, data [
sort.SliceStable(data, less)
}
func (c memoryExecutorContext) groupBy(selectStmt parsers.SelectStmt, data []RowType) []RowType {
groupedRows := make(map[string][]RowType)
groupedKeys := make([]string, 0)
// Group rows by group by columns
for _, row := range data {
key := c.generateGroupKey(selectStmt.GroupBy, row)
if _, ok := groupedRows[key]; !ok {
groupedKeys = append(groupedKeys, key)
}
groupedRows[key] = append(groupedRows[key], row)
}
// Aggregate each group
aggregatedRows := make([]RowType, 0)
for _, key := range groupedKeys {
groupRows := groupedRows[key]
aggregatedRow := c.aggregateGroup(selectStmt, groupRows)
aggregatedRows = append(aggregatedRows, aggregatedRow)
}
return aggregatedRows
}
func (c memoryExecutorContext) generateGroupKey(groupByFields []parsers.SelectItem, row RowType) string {
var keyBuilder strings.Builder
for _, column := range groupByFields {
fieldValue := c.getFieldValue(column, row)
keyBuilder.WriteString(fmt.Sprintf("%v", fieldValue))
keyBuilder.WriteString(":")
}
return keyBuilder.String()
}
func (c memoryExecutorContext) aggregateGroup(selectStmt parsers.SelectStmt, groupRows []RowType) RowType {
aggregatedRow := c.selectRow(selectStmt.SelectItems, groupRows)
return aggregatedRow
}
func compareValues(val1, val2 interface{}) int {
if reflect.TypeOf(val1) != reflect.TypeOf(val2) {
return 1
@@ -349,3 +430,43 @@ func compareValues(val1, val2 interface{}) int {
return 1
}
}
func deduplicate(slice []RowType) []RowType {
var result []RowType
for i := 0; i < len(slice); i++ {
unique := true
for j := 0; j < len(result); j++ {
if compareValues(slice[i], result[j]) == 0 {
unique = false
break
}
}
if unique {
result = append(result, slice[i])
}
}
return result
}
func hasAggregateFunctions(selectItems []parsers.SelectItem) bool {
if selectItems == nil {
return false
}
for _, selectItem := range selectItems {
if selectItem.Type == parsers.SelectItemTypeFunctionCall {
if typedValue, ok := selectItem.Value.(parsers.FunctionCall); ok && slices.Contains[[]parsers.FunctionCallType](parsers.AggregateFunctions, typedValue.Type) {
return true
}
}
if hasAggregateFunctions(selectItem.SelectItems) {
return true
}
}
return false
}

View File

@@ -59,6 +59,26 @@ func Test_Execute(t *testing.T) {
)
})
t.Run("Should execute SELECT with GROUP BY", func(t *testing.T) {
testQueryExecute(
t,
parsers.SelectStmt{
SelectItems: []parsers.SelectItem{
{Path: []string{"c", "pk"}},
},
Table: parsers.Table{Value: "c"},
GroupBy: []parsers.SelectItem{
{Path: []string{"c", "pk"}},
},
},
mockData,
[]memoryexecutor.RowType{
map[string]interface{}{"pk": 123},
map[string]interface{}{"pk": 456},
},
)
})
t.Run("Should execute IN function", func(t *testing.T) {
testQueryExecute(
t,

View File

@@ -35,6 +35,24 @@ func Test_Execute_Select(t *testing.T) {
)
})
t.Run("Should execute SELECT DISTINCT", func(t *testing.T) {
testQueryExecute(
t,
parsers.SelectStmt{
SelectItems: []parsers.SelectItem{
{Path: []string{"c", "pk"}},
},
Table: parsers.Table{Value: "c"},
Distinct: true,
},
mockData,
[]memoryexecutor.RowType{
map[string]interface{}{"pk": 123},
map[string]interface{}{"pk": 456},
},
)
})
t.Run("Should execute SELECT TOP", func(t *testing.T) {
testQueryExecute(
t,
@@ -53,6 +71,32 @@ func Test_Execute_Select(t *testing.T) {
)
})
t.Run("Should execute SELECT OFFSET", func(t *testing.T) {
testQueryExecute(
t,
parsers.SelectStmt{
SelectItems: []parsers.SelectItem{
{Path: []string{"c", "id"}},
{Path: []string{"c", "pk"}},
},
Table: parsers.Table{Value: "c"},
Count: 2,
Offset: 1,
OrderExpressions: []parsers.OrderExpression{
{
SelectItem: parsers.SelectItem{Path: []string{"c", "id"}},
Direction: parsers.OrderDirectionDesc,
},
},
},
mockData,
[]memoryexecutor.RowType{
map[string]interface{}{"id": "67890", "pk": 456},
map[string]interface{}{"id": "456", "pk": 456},
},
)
})
t.Run("Should execute SELECT VALUE", func(t *testing.T) {
testQueryExecute(
t,

View File

@@ -4,6 +4,7 @@ import (
"fmt"
"strings"
"github.com/pikami/cosmium/internal/logger"
"github.com/pikami/cosmium/parsers"
)
@@ -118,7 +119,7 @@ func (c memoryExecutorContext) strings_Left(arguments []interface{}, row RowType
lengthEx := c.getFieldValue(arguments[1].(parsers.SelectItem), row)
if length, ok = lengthEx.(int); !ok {
fmt.Println("strings_Left - got parameters of wrong type")
logger.Error("strings_Left - got parameters of wrong type")
return ""
}
@@ -157,7 +158,7 @@ func (c memoryExecutorContext) strings_Replicate(arguments []interface{}, row Ro
timesEx := c.getFieldValue(arguments[1].(parsers.SelectItem), row)
if times, ok = timesEx.(int); !ok {
fmt.Println("strings_Replicate - got parameters of wrong type")
logger.Error("strings_Replicate - got parameters of wrong type")
return ""
}
@@ -190,7 +191,7 @@ func (c memoryExecutorContext) strings_Right(arguments []interface{}, row RowTyp
lengthEx := c.getFieldValue(arguments[1].(parsers.SelectItem), row)
if length, ok = lengthEx.(int); !ok {
fmt.Println("strings_Right - got parameters of wrong type")
logger.Error("strings_Right - got parameters of wrong type")
return ""
}
@@ -219,11 +220,11 @@ func (c memoryExecutorContext) strings_Substring(arguments []interface{}, row Ro
lengthEx := c.getFieldValue(arguments[2].(parsers.SelectItem), row)
if startPos, ok = startPosEx.(int); !ok {
fmt.Println("strings_Substring - got start parameters of wrong type")
logger.Error("strings_Substring - got start parameters of wrong type")
return ""
}
if length, ok = lengthEx.(int); !ok {
fmt.Println("strings_Substring - got length parameters of wrong type")
logger.Error("strings_Substring - got length parameters of wrong type")
return ""
}
@@ -263,7 +264,7 @@ func (c memoryExecutorContext) parseString(argument interface{}, row RowType) st
return str1
}
fmt.Println("StringEquals got parameters of wrong type")
logger.Error("StringEquals got parameters of wrong type")
return ""
}