Implementation of EOF in Solidity #15294

rodiazet · 2024-07-23T16:52:24Z

EIP-7480 (Only DATALOADN)
EIP-4200 (w/o RJUMPV)
EIP-4750
EIP-6206
EIP-7620
EIP-7069
EIP-5450
evmc update to 12
Change sendMessage interface
Add compileToEOF flag in semantic tests settings
Semantic tests updated

TODO:

There is a couple TODOs added in the commits. They need to be implemented in separated PR.
~~Immutables bug reported by Paradigm needs to be fixed.~~ FIXED
Semantic tests may stop passing because part of them depend on exact contract code which may change with compiler development. Additional support for contract addresses needs to be implemented in soltest.
CI integration may be required.

github-actions · 2024-07-23T16:52:36Z

Thank you for your contribution to the Solidity compiler! A team member will follow up shortly.

If you haven't read our contributing guidelines and our review checklist before, please do it now, this makes the reviewing process and accepting your contribution smoother.

If you have any questions or need our help, feel free to post them in the PR or talk to us directly on the #solidity-dev channel on Matrix.

nikola-matic · 2024-07-30T10:17:21Z

libevmasm/Assembly.cpp

-		m_items.back().setLocation(m_currentSourceLocation);
-	m_items.back().m_modifierDepth = m_currentModifierDepth;
-	return m_items.back();
+	auto& currentItems = m_codeSections.at(m_currentCodeSection).items;


m_codeSection.at(...) can throw - either a util::contains check should go here, or even better, an assertion if you're sure that the current code section is present.

Right. I'm adding an assertion as m_currentCodeSection is changed on in void beginFunction(uint16_t _functionID) and similar assertion is there too.

nikola-matic · 2024-07-30T10:22:05Z

libevmasm/Assembly.cpp

 }

 unsigned Assembly::codeSize(unsigned subTagSize) const
 {
 	for (unsigned tagSize = subTagSize; true; ++tagSize)
 	{
 		size_t ret = 1;
-		for (auto const& i: m_data)


Is m_data no longer taken into account?

That part of the calculation moved to ::assemble - it's part of bytesRequiredForDataUpperBound there (both for EOF and legacy)...

But we should probably rethink this as well at the latest as part of #15294 (comment) - For example: since in EOF there won't be any PushTags, the weird outer loop here becomes superfluous as well, since code size no longer depends on the tag size (the "tag size" in general becomes irrelevant).

For now, maybe a comment in the header is enough, clarifying that even for legacy this now only counts "pure" code size without data?

libevmasm/Assembly.cpp

nikola-matic · 2024-07-30T16:34:06Z

libevmasm/Assembly.cpp

+
+		std::vector<size_t> successors;
+
+		if (


I assume that ConditionalRelativeJump is not here because both 'branches' need to have their stack heights calculated? I.e. both the next assembly item as well as the tagged one that we're (possibly) jumping to?

Yes. For ConditionalRelativeJump we need to calculated both branches and one of them starts with the next instruction but second is the jump target. It's being added to successors by next if

nikola-matic · 2024-07-30T16:38:57Z

libevmasm/Assembly.cpp

+{
+	uint16_t maxStackHeight = _args;
+	std::stack<size_t> worklist;
+	std::vector<int32_t> stack_heights(_items.size(), -1);


Suggested change

std::vector<int32_t> stack_heights(_items.size(), -1);

std::vector<size_t> stack_heights(_items.size(), std::numeric_limits<size_t>::max);

stackHeight should also be unsigned - I would assume our stack height is never going to be negative?

libevmasm/Assembly.cpp

libevmasm/BlockDeduplicator.cpp

ekpyron · 2024-07-31T21:06:33Z

libevmasm/Assembly.cpp

@@ -396,7 +399,8 @@ void Assembly::assemblyStream(
 {
 	Functionalizer f(_out, _prefix, _sourceCodes, *this);

-	for (auto const& i: m_items)
+	// TODO: support EOF


In cases like this (and e.g. similarly for Assembly::assemblyJSON and possibly a few of other similar TODOs like this), it'd be better to add an explicit

solUnimplementedAssert(!m_eofVersion.has_value(), "Assembly output for EOF is not yet implemented.");

Better to error out and then implement it properly later in a subsequent PR than to give a partial output of only the first code section with the rest missing.

Done. Additionally I switched off the assembly optimization for EOF as event first step requires verification.

…typo in input parsing Co-authored-by: Daniel Kirchner <[email protected]>

Co-authored-by: Daniel Kirchner <[email protected]>

…EIP-3540) Co-authored-by: Daniel Kirchner <[email protected]>

Add `DATALOAD` instruction

Co-authored-by: Daniel Kirchner <[email protected]>

Preparation to implement new contract creation Co-authored-by: Daniel Kirchner <[email protected]>

Co-authored-by: Daniel Kirchner <[email protected]>

…endMessage Co-authored-by: Rodrigo Q. Saramago <[email protected]>

libsolidity/experimental/codegen/IRGenerator.cpp

ekpyron · 2024-08-22T15:05:45Z

libyul/backends/evm/ControlFlowGraphBuilder.cpp

@@ -76,6 +87,16 @@ void cleanUnreachable(CFG& _cfg)
 		cxx20::erase_if(node->entries, [&](CFG::BasicBlock* entry) -> bool {
 			return !reachabilityCheck.visited.count(entry);
 		});
+
+	// Remove functions which are never referenced.


This change in cleanUnreachable could be split out into a separate PR and merged directly into develop up front, right?
EDIT: we can revisit once we get to the commits containing it, whether splitting really makes things easier, fine for now.

ekpyron · 2024-08-22T15:21:28Z

libsolidity/interface/StandardCompiler.cpp

-		auto eofVersion = settings["evmVersion"].get<uint8_t>();
+		auto eofVersion = settings["eofVersion"].get<uint8_t>();


This and the related evm-version check change can and should separately go into develop :-).
EDIT: as per chat, let's just merge the first three commits for now :-).

ekpyron · 2024-08-22T16:08:43Z

libsolidity/interface/CompilerStack.cpp

@@ -1412,7 +1420,7 @@ void CompilerStack::compileContract(

 	Contract& compiledContract = m_contracts.at(_contract.fullyQualifiedName());

-	std::shared_ptr<Compiler> compiler = std::make_shared<Compiler>(m_evmVersion, m_revertStrings, m_optimiserSettings);
+	std::shared_ptr<Compiler> compiler = std::make_shared<Compiler>(m_evmVersion, m_eofVersion, m_revertStrings, m_optimiserSettings);


Instead of passing m_eofVersion around here, we can just assert

solUnimplementedAssert(!m_eofVersion, "EOF is not supported for legacy code generation");

(this is only-legacy, see the solAssert(!m_viaIR, ""); below)
And thereby leave libsolidity/codegen/Compiler.h and libsolidity/codegen/CompilerContext.h largely untouched (resp. where they need an eof version just use std::nullopt directly).

Addressed in https://github.com/ethereum/solidity/pull/15356/files#diff-0f19df92d55a7d7fe38fdf4be1e14b364671c4934853d13a33b31aedf3f917bdR1424

cairoeth · 2024-08-23T05:13:48Z

libevmasm/AssemblyItem.cpp

+		text = "eofcreate(" +  std::to_string(static_cast<size_t>(data())) + ")";
+		break;
+	case ReturnContract:
+		text = "returcontract(" +  std::to_string(static_cast<size_t>(data())) + ")";


typo in assembly text of RETURNCONTRACT

Suggested change

text = "returcontract(" + std::to_string(static_cast<size_t>(data())) + ")";

text = "returncontract(" + std::to_string(static_cast<size_t>(data())) + ")";

github-actions bot added the external contribution ⭐ label Jul 23, 2024

rodiazet force-pushed the eof-cleaned branch 10 times, most recently from c49b88b to 88cf606 Compare July 29, 2024 16:56

nikola-matic reviewed Jul 31, 2024

View reviewed changes

ekpyron reviewed Jul 31, 2024

View reviewed changes

ekpyron mentioned this pull request Jul 31, 2024

Refine and Stabilize EOF Support #15310

Open

37 tasks

rodiazet force-pushed the eof-cleaned branch 2 times, most recently from 5ab6923 to 64e4050 Compare August 8, 2024 07:57

rodiazet and others added 13 commits August 8, 2024 15:14

prep: Extend StandardCompiler settings with eofVersion flag. Fix …

4ae6d1c

…typo in input parsing Co-authored-by: Daniel Kirchner <[email protected]>

prep: Pass _eofVersion to libevmasm/Assembly class

edc3b40

Co-authored-by: Daniel Kirchner <[email protected]>

prep: Change toBigEndian interface to support rvalue ref.

07528fc

Co-authored-by: Daniel Kirchner <[email protected]>

eof: Introduce EOF container format support in Assembly::assemble (…

c0a839e

…EIP-3540) Co-authored-by: Daniel Kirchner <[email protected]>

eof: Support DATALOADN (EIP-7480)

84ab885

Add `DATALOAD` instruction

eof: Support RJUMP, RJUMPI (EIP-4200)

70b9e8b

Co-authored-by: Daniel Kirchner <[email protected]>

eof: Support CALLF, JUMPF, RETF. (EIP-4750, EIP-6206)

64cbc1d

Co-authored-by: Daniel Kirchner <[email protected]>

eof: Pass flag to copyConstructorArgumentsToMemoryFunction (EIP-7620)

412edcd

Preparation to implement new contract creation Co-authored-by: Daniel Kirchner <[email protected]>

eof: Implement EOF Contract Creation (EIP-7620)

76a25cc

eof: Implement revamped CALL instructions (EIP-7069)

a419908

eof: Implement max stack heigh calculation (EIP-5450)

79b4d65

Co-authored-by: Daniel Kirchner <[email protected]>

prep: Update to emvc 12

bdc126f

tests: Add new (EOF) message kind to EVMHost. Separate arguments in s…

00b0b14

…endMessage Co-authored-by: Rodrigo Q. Saramago <[email protected]>

rodiazet added 2 commits August 8, 2024 15:40

tests: Add compileToEOF flag to semantic tests settings

a902291

tests: Update semantic tests for EOF

bbfb85d

rodiazet force-pushed the eof-cleaned branch from 64e4050 to bbfb85d Compare August 8, 2024 13:46

ekpyron reviewed Aug 22, 2024

View reviewed changes

libsolidity/experimental/codegen/IRGenerator.cpp Show resolved Hide resolved

ekpyron reviewed Aug 22, 2024

View reviewed changes

cairoeth reviewed Aug 23, 2024

View reviewed changes

cameel added the EOF label Aug 29, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implementation of EOF in Solidity #15294

Implementation of EOF in Solidity #15294

rodiazet commented Jul 23, 2024 •

edited

Loading

github-actions bot commented Jul 23, 2024

nikola-matic Jul 30, 2024

rodiazet Aug 7, 2024

nikola-matic Jul 30, 2024

ekpyron Jul 31, 2024

nikola-matic Jul 30, 2024

rodiazet Aug 8, 2024

nikola-matic Jul 30, 2024

rodiazet Aug 8, 2024

ekpyron Jul 31, 2024

rodiazet Aug 7, 2024 •

edited

Loading

ekpyron Aug 22, 2024 •

edited

Loading

ekpyron Aug 22, 2024 •

edited

Loading

ekpyron Aug 22, 2024 •

edited

Loading

rodiazet Aug 23, 2024

cairoeth Aug 23, 2024

	std::vector<int32_t> stack_heights(_items.size(), -1);
	std::vector<size_t> stack_heights(_items.size(), std::numeric_limits<size_t>::max);

		auto eofVersion = settings["evmVersion"].get<uint8_t>();
		auto eofVersion = settings["eofVersion"].get<uint8_t>();

	text = "returcontract(" + std::to_string(static_cast<size_t>(data())) + ")";
	text = "returncontract(" + std::to_string(static_cast<size_t>(data())) + ")";

Implementation of EOF in Solidity #15294

Are you sure you want to change the base?

Implementation of EOF in Solidity #15294

Conversation

rodiazet commented Jul 23, 2024 • edited Loading

github-actions bot commented Jul 23, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rodiazet Aug 7, 2024 • edited Loading

Choose a reason for hiding this comment

ekpyron Aug 22, 2024 • edited Loading

Choose a reason for hiding this comment

ekpyron Aug 22, 2024 • edited Loading

Choose a reason for hiding this comment

ekpyron Aug 22, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rodiazet commented Jul 23, 2024 •

edited

Loading

rodiazet Aug 7, 2024 •

edited

Loading

ekpyron Aug 22, 2024 •

edited

Loading

ekpyron Aug 22, 2024 •

edited

Loading

ekpyron Aug 22, 2024 •

edited

Loading