Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Paloma: Optimize Endlbock marshaling to remove yaml marshaling #265

Closed
taariq opened this issue May 30, 2023 · 4 comments
Closed

Paloma: Optimize Endlbock marshaling to remove yaml marshaling #265

taariq opened this issue May 30, 2023 · 4 comments
Assignees
Labels
enhancement New feature or request paloma

Comments

@taariq
Copy link
Member

taariq commented May 30, 2023

Comment from @tac0turtle: "you guys are doing some crazy marhsaling in your endblock. it would be good to remove that or change it to something efficient. Not sure how these to things interact but if just not use yaml marshaling here."

Requirements:

  • Refactor Endlbock to remove yaml marshaling

Image
Image

@byte-bandit
Copy link

@taariq @ToasterTheBrave I have a fix for this one ready, but it does come with the disadvantage of diluting the logging. There are lots of pointer fields on this message, and with the yaml marshalling, I assume the information was preserved. There's no way that I'm aware of to automatically dereference pointers using a regular fmt.Sprintf() call.

What is the amount of information we actually want to log here? Is there a subset of information that we can log instead?

@ToasterTheBrave
Copy link

@klausklapper Do you have a quick example of what the logs look like before and after?

@byte-bandit
Copy link

@ToasterTheBrave Most of the the log message consists of byte array slices and or pointer addresses, both with and without YAML marshalling, it's just formatted differently.

I'd argue that logging the entire message is superfluous, as we're able to retrieve the contents directly from Paloma anyway using the message ID, which we already log. Unless there is some benefit of pumping all message contents into the log, I will go ahead and remove the log field from all calls in Paloma & Pigeon.

byte-bandit added a commit that referenced this issue Jul 4, 2023
Logging the entire message in byte code not only take quite a long
time due to marshalling, it's also largely irrelevant, as message
information can easily be obtained from the queue.
byte-bandit added a commit that referenced this issue Jul 4, 2023
Logging the entire message in byte code not only take quite a long
time due to marshalling, it's also largely irrelevant, as message
information can easily be obtained from the queue.
@taariq taariq closed this as completed Jul 4, 2023
@taariq
Copy link
Member Author

taariq commented Jul 4, 2023

Fixed by palomachain#897

byte-bandit added a commit that referenced this issue Jul 5, 2023
Logging the entire message in byte code not only take quite a long
time due to marshalling, it's also largely irrelevant, as message
information can easily be obtained from the queue.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request paloma
Projects
None yet
Development

No branches or pull requests

3 participants