Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update tokio version #13

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[workspace]

members = [
"clickhouse-driver","cityhash","lz4a","derive","cityhash-rs"
"clickhouse-driver","cityhash","derive","cityhash-rs"
]

[profile.release]
Expand Down
89 changes: 44 additions & 45 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@
[![Build Status](https://travis-ci.org/ddulesov/clickhouse_driver.svg?branch=master)](https://travis-ci.org/ddulesov/clickhouse_driver)
![Rust](https://github.com/ddulesov/clickhouse_driver/workflows/Rust/badge.svg?branch=master)

Asynchronous pure rust tokio-based Clickhouse client library
development status: **alpha**
Asynchronous pure rust tokio-based Clickhouse client library

development status: **alpha**

Tested on Linux x86-64 (ubuntu 20.04 LTS), Windows 10.

Expand Down Expand Up @@ -32,8 +32,8 @@ Tested on Linux x86-64 (ubuntu 20.04 LTS), Windows 10.

### Use cases ###

* Make query using SQL syntax supported by Clickhouse Server
* Execute arbitrary DDL commands
* Make query using SQL syntax supported by Clickhouse Server
* Execute arbitrary DDL commands
* Query Server status
* Insert into Clickhouse Server big (possibly continues ) data stream
* Load-balancing using round-robin method
Expand All @@ -50,21 +50,20 @@ git module update --init --recursive
Building requires rust 1.41 stable or nightly,
tokio-0.2.x.

- Add next lines into the `dependencies` section of your `Cargo.toml`:
- Add next lines into the `dependencies` section of your `Cargo.toml`:

```toml
```toml
clickhouse-driver = { version="0.1.0-alpha.3", path="../path_to_package/clickhouse-driver"}
clickhouse-driver-lz4 = { version="0.1.0", path="../path_to_package/lz4a"}
clickhouse-driver-cthrs = { version="0.1.0", path="../path_to_package/cityhash-rs"}

```
- Add usage in main.rs
```rust
extern crate clickhouse_driver;
extern crate clickhouse_driver;
use clickhouse_driver::prelude::*;
```
to connect to server provide connection url

to connect to server provide connection url
```
tcp://username:password@localhost/database?paramname=paramvalue&...
```
Expand All @@ -77,65 +76,65 @@ tcp://user:default@localhost/log?ping_timout=200ms&execute_timeout=5s&query_time
lz4 - fast and efficient compression method.
It can significantly reduce transmitted data size and time if used for
big data chunks. For small data it's better to choose none compression;

* `connection_timeout` - timeout for establishing connection.
Default is 500ms;

* `execute_timeout` - timeout for waiting result of **execute** method call
If the execute used for alter huge table it can take
If the execute used for alter huge table it can take
long time to complete. In this case set this parameter to appropriate
value. In other cases leave the default value (180 sec);

* `query_timout` - timeout for waiting response from the server with
next block of data in **query** call.
Note. Large data query may take long time. This timeout requires that only
Note. Large data query may take long time. This timeout requires that only
one chunk of data will receive until the end of timeout.
Default value is 180sec;

* `insert_timeout` - timeout for waiting result of `insert` call
insert method call returns error if the server does not receive
message until the end of insert_timeout.
As insert data processing is asynchronous it doesn't include server block processing time.
Default value is 180 sec;

* `ping_timout` - wait before ping response.
The host will be considered unavailable if the server
does not return pong response until the end of ping_timeout;
* `retry_timeout` - the number of seconds to wait before send next ping
if the server does not return;
* `ping_before_query` - 1 (default) or 0. This option if set
requires the driver to check Clickhouse server responsibility
The host will be considered unavailable if the server
does not return pong response until the end of ping_timeout;

* `retry_timeout` - the number of seconds to wait before send next ping
if the server does not return;

* `ping_before_query` - 1 (default) or 0. This option if set
requires the driver to check Clickhouse server responsibility
after returning connection from pool;
* `pool_min` - minimal connection pool size.

* `pool_min` - minimal connection pool size.
the number of idle connections that can be kept in the pool;
Default value is 2;
* `pool_max` - maximum number of established connections that the pool

* `pool_max` - maximum number of established connections that the pool
can issued. If the task require new connection while the pool reaches the maximum
and there is not idle connection then this task will be put in waiting queue.
Default value is 10;
* `readonly` - 0 (default) |1|2.
0 - all commands allowed.
2- select queries and change settings,

* `readonly` - 0 (default) |1|2.
0 - all commands allowed.
2- select queries and change settings,
1 - only select queries ;

* `keepalive` - keepalive TCP option;

* `host` - alternative host(s)

All timeout parameters accept integer number - the number of seconds.
To specify timeout in milliseconds add `ms` at the end.
Examples:
Examples:
- `200ms` ( 200 mseconds )
- `20` ( 20 seconds )
- `10s` ( 10 seconds )

### Example
```rust
```rust

struct Blob {
id: u64,
Expand Down Expand Up @@ -209,7 +208,7 @@ async fn main() -> Result<(), io::Error> {
* Doesn't support multidimensional Array,
* Array data types readonly
* LowCardinality - readonly and just String base type
* Insert method support only limited data types
* Insert method support only limited data types
`insert` requires that inserted data exactly matches table column type
- Int8(16|32|64) - i8(16|32|64)
- UInt8(16|32|64) - u8(16|32|64)
Expand All @@ -222,16 +221,16 @@ async fn main() -> Result<(), io::Error> {
- IPv6 - AddrIpv6
- String - &str,String, or &[u8]
- Enum8|16 - &str or String. Also, i16 value of enum index can be retrieved.

### Roadmap

* `Array` column data type - read/write
* `Tuple` - no plans to support
* `Array` column data type - read/write
* `Tuple` - no plans to support
* `AggregateFunction` - no plans to support
* `LowCardinality` - add write support, extend it to `Date`, `DateTime` types
* `LowCardinality` - add write support, extend it to `Date`, `DateTime` types
* `Serde` - Row serializer/deserializer interface in addition to ad-hoc one
* `TLS`
* C-API ?
* `async_std` runtime


8 changes: 2 additions & 6 deletions cityhash-rs/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -298,7 +298,6 @@ pub fn city_hash_128(src: &[u8]) -> Pair {
mod tests {
use super::*;
use clickhouse_driver_cth::{_CityHash128, _CityMurmur, c_char, Hash128};
use std::vec::Vec;

fn city_hash_ref(source: &[u8]) -> Pair {
let h = unsafe { _CityHash128(source.as_ptr() as *const c_char, source.len()) };
Expand Down Expand Up @@ -392,14 +391,11 @@ mod tests {

#[test]
fn test_hash_128() {
const MAX_SIZE: u32 = 1024 * 10;
const MAX_SIZE: u32 = 1024;
const ITER_COUNT: u8 = 5;
use rand::Rng;
for s in 8..MAX_SIZE {
let mut b: Vec<u8> = Vec::with_capacity(s as usize);
unsafe {
b.set_len(s as usize);
}
let mut b = std::vec![0u8; s as usize];
for _ in 0..ITER_COUNT {
rand::thread_rng().fill(&mut b[..]);
assert_eq!(city_hash_ref(b.as_ref()), city_hash_128(b.as_ref()));
Expand Down
5 changes: 3 additions & 2 deletions clickhouse-driver/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,13 @@ repository = "https://github.com/ddulesov/clickhouse_driver"
[dependencies]
clickhouse-driver-cth = { version="0.1.0", path="../cityhash", optional=true}
clickhouse-driver-cthrs = { version="0.1.1", path="../cityhash-rs", optional=true}
clickhouse-driver-lz4 = { version="0.1.0", path="../lz4a" }

lz4 = "1.23.1"
thiserror = { version="1.0" }
log = { version="0.4.8" }
url = { version="^2" }
tokio = { version = "0.2", features = ["rt-core", "sync", "tcp", "time", "dns", "stream", "test-util", "io-util","macros"] }
tokio = { version = "1.15.0", features = ["rt", "sync", "net", "time", "rt-multi-thread", "test-util", "io-util","macros"] }

tokio-native-tls = { version = "0.1.0", optional= true }
pin-project-lite = { version="^0.1" }
futures = { version="0.3" }
Expand Down
5 changes: 3 additions & 2 deletions clickhouse-driver/examples/array.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,10 @@ use std::{env, io, time};

macro_rules! get {
($row: ident, $i: expr, $err: ident) => {
$row.value($i)?.ok_or_else($err)?;
$row.value($i)?.ok_or_else($err)?
};
($row: ident, $i: expr, $err: ident, opt) => {
$row.value($i)?;
$row.value($i)?
};
}
// CREATE table mainx(
Expand Down Expand Up @@ -64,6 +64,7 @@ async fn main() -> Result<(), io::Error> {
}
}
eprintln!("fetch {} rows in {} msec", c, start.elapsed().as_millis());
eprintln!("progress {:?}", result.progress);
}

Ok(())
Expand Down
2 changes: 1 addition & 1 deletion clickhouse-driver/examples/bulk-insert.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ static NAMES: [&str; 5] = ["one", "two", "three", "four", "five"];
/// Block size
const BSIZE: u64 = 10000;
/// The number of blocks
const CIRCLE: u64 = 1000;
const CIRCLE: u64 = 10;

fn next_block(i: u64) -> Block<'static> {
let now = chrono::offset::Utc::now();
Expand Down
2 changes: 1 addition & 1 deletion clickhouse-driver/examples/insert-select.rs
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ impl Deserialize for Blob {
})
}
}
const C: u64 = 10000;
const C: u64 = 100;

#[tokio::main]
async fn main() -> Result<(), io::Error> {
Expand Down
2 changes: 1 addition & 1 deletion clickhouse-driver/examples/select.rs
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ struct Perf {

macro_rules! get {
($row: ident, $i: expr, $err: ident) => {
$row.value($i)?.ok_or_else($err)?;
$row.value($i)?.ok_or_else($err)?
};
}

Expand Down
Loading