CANNOT_READ_ALL_DATA when calling end() #109
Comments
|
I tested some of
{
"timestamp":1718746919678
"ip4":2130706433
"user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36"
"req_size":0
"content_length":0
"content_type":"image/x-icon"
"method":1
"host":"localhost"
"service":"favicon.ico"
"path":"/"
"status_code":200
"execution_time_ms":0
"version":"HTTP/1.1"
}
{
"timestamp":1718788113783
"api_key":"<my-key>"
"ip4":2130706433
"auth":true
"user_agent":"insomnia/9.2.0"
"req_size":0
"req_content_type":"application/json"
"content_length":4242
"content_type":"application/json"
"method":1
"host":"localhost"
"service":"my-service"
"path":"/"
"query":"f1=true&f2=true"
"status_code":200
"execution_time_ms":77
"version":"HTTP/1.1"
} |
Ok. I managed it to fix only in this way! the other ones don't seem to work |
|
maybe, it is better to close the issue only after some of your checks |
|
Hi, FWIW -- trying this against Clickhouse Cloud. I have changed my struct to make sure there are no options, and all values are populated, even thought the table allows most to be null. Ideas on where to look would be greatly appreciated. |
|
as an update, I was using chrono for DateTime and after seeing this issue and re-reading that time support, I switched over from chrono to time and still have same issue. |
|
Hi @apsaltis ! If you have any example snippet that can reproduce the actual data structures, please send here. Just to compare with those of mine and with all the attempts I did 😁 |
|
Hi @riccardogabellone -- here is the data structure:
I also just tried with crate feature "wa-37420" |
|
just changed the "raw_results" which was a JSON string to be just.a string literal "some_data" and now got back a different error: so now row 2 and not 13, not sure where that leads. |
at first glance, I note |
|
how is the corresponding clickhouse table declared? |
|
Sorry, should have included that:
|
|
Some checks I'd do:
|
Everything is working right now like a charm. For certian if I have the wrong serde for the datetime it blows up as expected. Thanks for all the help @riccardogabellone |
|
Hi @riccardogabellone , For now, I changed my table to be an Array(LC(String)) and it works just fine with a Vec Thoughts on the Vec? Thanks, |
|
Hm, I think it's easy to implement It looks clumsy; I'm sure there is a better approach here by changing pub fn serialize<S>(uuid: &Uuid, serializer: S) -> Result<S::Ok, S::Error>into something like pub fn serialize<U: Wrapper<Uuid>, S>(uuid: &U, serializer: S) -> Result<S::Ok, S::Error> |
|
Sounds good @loyd -- let me take a stab at implementing this and submit a pr. |
|
Hello, I have a similar issue here as well, here is what I sent to clickhouse and they said it was more likely an issue in the rust client. Do you want a separate issue @loyd ? Company or project name ACUTE Project, plenty of data inside the database (5.6B records so far). Describe the unexpected behaviour I'm trying to insert a few records into a table with the Rust client (https://docs.rs/clickhouse) and it generates the following error every time, usually on the 3rd or 4th records (quite small). How to reproduce (version 24.5.3.5 (official build)) Using the HTTP interface through the Rust client.
let r = r##"
CREATE TABLE ids (
drone_id VARCHAR,
callsign VARCHAR,
journey INT,
en_id VARCHAR,
) ENGINE = Memory
##;The code later uses this to insert records: #[derive(Clone, Debug, Serialize, Deserialize, Row)]
struct Tc {
#[serde(skip_deserializing)]
en_id: String,
journey: u32,
drone_id: String,
callsign: String,
}
let total = dbh.query("SELECT count() FROM today_close").fetch_one::<usize>().await?;
let r = format!(r##"
SELECT
journey,
drone_id,
callsign,
FROM today_close
WHERE
dist_drone_plane < 1852
GROUP BY ALL
"##);
trace!("Fetch close encounters from today_close.");
let all = dbh.query(&r).fetch_all::<Tc>().await?;
if all.len() == 0 {
return Err(eyre!("No encounters found out of {total}").into());
}
trace!("Insert updated records.");
// Insert the records
//
let mut batch = dbh.insert("acute.ids")?;
trace!("Add en_id.");
all.iter()
.enumerate()
.for_each(|(id, elem): (usize, &Tc)| {
let journey = elem.journey;
let elem = Tc {
en_id: format!("{}-{}-{}-{}", site, day_name, journey, id),
journey: elem.journey,
drone_id: elem.drone_id.clone(),
callsign: elem.callsign.clone(),
};
debug!("{elem:?}"); // DISPLAYS THE DEBUG LINES BELOW
block_on(async { batch.write(&elem).await.unwrap(); });
});
let _ = batch.end().await?; // CRASH IS HERE -- LINE 358I tried with Client::insert and the inserter (Client::Inserter).
Expected behavior Error message and/or stacktrace Additional context Having I don't think I can use a single |
|
I found what is bothering the client: the serde directive |
|
@keltia, would you like to open a separate issue? The client should generate a different subset of field names for serialize and deserialize processes. Now, it skips it in both cases if any |
Hi! I'm having this kind of issue, already encountered by someone else:
ERROR bad response: Code: 33. DB::Exception: Cannot read all data. Bytes read: 38. Bytes expected: 116.: (at row 1) : While executing BinaryRowInputFormat. (CANNOT_READ_ALL_DATA) (version 24.2.2.16288 (official build))I cannot figure out what is going wrong...
Here are my rust code:
And the clickhouse DB:
Maybe I missed something from docs?
Could be any of those
Option<_>that are not annotated in any way (despite I guessskip_serializing_ifshould be enough)?Or, do I need to put
Nullablefor each column to match rust impl? as I test, CH put default values anyway, so, as long as they are skipped, there should not be any exception, right?Or, is it better to put serde defaults without
Option<_>wrappers?EDIT: I also tried the
wa-37420feature flag. Same resultsThe text was updated successfully, but these errors were encountered: