Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reorg builtins #1847

Merged
merged 67 commits into from
Oct 1, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
e4c35fe
Experimental pil to rust compiler.
chriseth Feb 29, 2024
686bdc6
Compile and dlopen.
chriseth Aug 3, 2024
b2f8d88
oeu
chriseth Aug 3, 2024
392afc1
oeu
chriseth Aug 3, 2024
6b2029a
loaded sym
chriseth Aug 3, 2024
7ee9afc
log time
chriseth Aug 3, 2024
44e80b8
new crate.
chriseth Sep 12, 2024
65e5f44
fix
chriseth Sep 12, 2024
ec40700
work
chriseth Sep 12, 2024
0be7ac0
work
chriseth Sep 12, 2024
6aa9f45
wor
chriseth Sep 12, 2024
01272e8
loading
chriseth Sep 12, 2024
249f3c8
fix
chriseth Sep 12, 2024
1570fb0
sqrt
chriseth Sep 12, 2024
81dca2b
add benchmark
chriseth Sep 12, 2024
641f1f2
forgot test file
chriseth Sep 12, 2024
65c6c04
clean.
chriseth Sep 13, 2024
1578b1e
fix
chriseth Sep 13, 2024
ffa6187
Some logging.
chriseth Sep 13, 2024
51cae14
size in mb.
chriseth Sep 13, 2024
22de29a
Update pil-analyzer/tests/types.rs
chriseth Sep 13, 2024
2682d27
Use ibig.
chriseth Sep 13, 2024
e9d2910
Use native cpu.
chriseth Sep 13, 2024
541d8dc
clippy
chriseth Sep 19, 2024
306cbb2
Merge remote-tracking branch 'origin/main' into compi
chriseth Sep 23, 2024
5aea6b2
merge fix.
chriseth Sep 23, 2024
1a6db99
Remove ibig features.
chriseth Sep 23, 2024
5322ccd
Merge remote-tracking branch 'origin/main' into compi
chriseth Sep 23, 2024
cdf3ba9
Nicer error messages.
chriseth Sep 24, 2024
7b0dceb
Partial compile.
chriseth Sep 24, 2024
76e6389
clippy
chriseth Sep 24, 2024
00b56e6
fmt
chriseth Sep 24, 2024
3544d57
Update jit-compiler/src/lib.rs
chriseth Sep 24, 2024
5cf259c
Portability.
chriseth Sep 24, 2024
dc7e113
Merge branch 'compi' of ssh://github.com/powdr-labs/powdr into compi
chriseth Sep 24, 2024
e928a37
Update jit-compiler/tests/execution.rs
chriseth Sep 24, 2024
86b0a2d
fix error message.
chriseth Sep 24, 2024
3327e52
clippy
chriseth Sep 24, 2024
2a45653
Use extern c.
chriseth Sep 25, 2024
5647eb6
Use libloading.
chriseth Sep 25, 2024
540671d
Update jit-compiler/src/compiler.rs
chriseth Sep 25, 2024
2937860
Extract sqrt code.
chriseth Sep 25, 2024
681937a
Remove drop.
chriseth Sep 25, 2024
6a96b72
Add release - we need the variable.
chriseth Sep 25, 2024
8db0eba
Encapsulate temp dir in struct.
chriseth Sep 25, 2024
b47df41
Simplify compiler state.
chriseth Sep 25, 2024
bdc2d38
Error message.
chriseth Sep 25, 2024
8c23119
use unsafe extern C fn
chriseth Sep 25, 2024
9501ef9
use mebibytes.
chriseth Sep 25, 2024
13305d1
Support functions as expressions.
chriseth Sep 24, 2024
f5f84f3
Extract magic numbers.
chriseth Sep 26, 2024
bd515f1
Make function safe.
chriseth Sep 26, 2024
1641fb6
Merge branch 'compi' into functions_as_expressions
chriseth Sep 26, 2024
0e7d2b8
Merge remote-tracking branch 'origin/main' into functions_as_expressions
chriseth Sep 27, 2024
3e9c60e
Implement field element type.
chriseth Sep 27, 2024
e3dc589
Re-organize builtins.
chriseth Sep 27, 2024
d1ee4bd
Format numbers properly.
chriseth Sep 27, 2024
9ad8b18
Merge branch 'format_number' into reorg_builtins
chriseth Sep 27, 2024
2e371c8
Rename function and use single match.
chriseth Sep 27, 2024
05be1a4
typo
chriseth Sep 27, 2024
be0cd4f
Merge remote-tracking branch 'origin/main' into format_number
chriseth Sep 30, 2024
8cd3833
fix shift
chriseth Sep 30, 2024
a00d4f4
fix merge
chriseth Sep 30, 2024
d1d757e
Merge remote-tracking branch 'origin/main' into reorg_builtins
chriseth Sep 30, 2024
a7193ca
Merge remote-tracking branch 'origin/format_number' into reorg_builtins
chriseth Sep 30, 2024
3e4926f
remove parts from other pr
chriseth Sep 30, 2024
ef2be05
Merge remote-tracking branch 'origin/main' into reorg_builtins
chriseth Sep 30, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions jit-compiler/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ log = "0.4.18"
mktemp = "0.5.0"
itertools = "0.13"
libloading = "0.8"
lazy_static = "1.4.0"

[dev-dependencies]
powdr-pil-analyzer.workspace = true
Expand Down
58 changes: 38 additions & 20 deletions jit-compiler/src/codegen.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
use std::collections::HashMap;
use std::{collections::HashMap, sync::OnceLock};

use itertools::Itertools;
use powdr_ast::{
Expand Down Expand Up @@ -76,7 +76,7 @@ impl<'a, T: FieldElement> CodeGenerator<'a, T> {

fn generate_code(&mut self, symbol: &str) -> Result<String, String> {
if let Some(code) = try_generate_builtin::<T>(symbol) {
return Ok(code);
return Ok(code.clone());
}

let Some((_, Some(FunctionValueDefinition::Expression(value)))) =
Expand Down Expand Up @@ -297,7 +297,7 @@ impl<'a, T: FieldElement> CodeGenerator<'a, T> {
/// This is either the escaped name of the symbol or a deref operator
/// applied to it.
fn symbol_reference(&self, symbol: &str) -> String {
let needs_deref = if is_builtin(symbol) {
let needs_deref = if is_builtin::<T>(symbol) {
false
} else {
let (_, def) = self.analyzed.definitions.get(symbol).as_ref().unwrap();
Expand Down Expand Up @@ -354,25 +354,43 @@ fn map_type(ty: &Type) -> String {
}
}

fn is_builtin(symbol: &str) -> bool {
matches!(
symbol,
"std::check::panic" | "std::field::modulus" | "std::convert::fe"
)
fn get_builtins<T: FieldElement>() -> &'static HashMap<String, String> {
static BUILTINS: OnceLock<HashMap<String, String>> = OnceLock::new();
BUILTINS.get_or_init(|| {
[
(
"std::array::len",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is std::convert::fe missing on purpose?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was not working as it was written any way. Will be re-added in #1834

"<T>(a: Vec<T>) -> ibig::IBig { ibig::IBig::from(a.len()) }".to_string(),
),
(
"std::check::panic",
"(s: &str) -> ! { panic!(\"{s}\"); }".to_string(),
),
(
"std::field::modulus",
format!(
"() -> ibig::IBig {{ {} }}",
format_unsigned_integer(&T::modulus().to_arbitrary_integer())
),
),
]
.into_iter()
.map(|(name, code)| {
(
name.to_string(),
format!("fn {}{code}", escape_symbol(name)),
)
})
.collect()
})
}

fn try_generate_builtin<T: FieldElement>(symbol: &str) -> Option<String> {
let code = match symbol {
"std::array::len" => "<T>(a: Vec<T>) -> ibig::IBig { ibig::IBig::from(a.len()) }".to_string(),
"std::check::panic" => "(s: &str) -> ! { panic!(\"{s}\"); }".to_string(),
"std::field::modulus" => {
format!("() -> ibig::IBig {{ {} }}", format_unsigned_integer(&T::modulus().to_arbitrary_integer()))
}
"std::convert::fe" => "(n: ibig::IBig) -> FieldElement {\n <FieldElement as PrimeField>::BigInt::try_from(n.to_biguint().unwrap()).unwrap().into()\n}"
.to_string(),
_ => return None,
};
Some(format!("fn {}{code}", escape_symbol(symbol)))
fn is_builtin<T: FieldElement>(symbol: &str) -> bool {
get_builtins::<T>().contains_key(symbol)
}

fn try_generate_builtin<T: FieldElement>(symbol: &str) -> Option<&String> {
get_builtins::<T>().get(symbol)
}

#[cfg(test)]
Expand Down
8 changes: 7 additions & 1 deletion jit-compiler/tests/execution.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,13 @@ use powdr_pil_analyzer::analyze_string;

fn compile(input: &str, symbol: &str) -> LoadedFunction {
let analyzed = analyze_string::<GoldilocksField>(input).unwrap();
powdr_jit_compiler::compile(&analyzed, &[symbol]).unwrap()[symbol].clone()
powdr_jit_compiler::compile(&analyzed, &[symbol])
.map_err(|e| {
eprintln!("Error jit-compiling:\n{e}");
e
})
.unwrap()[symbol]
.clone()
}

#[test]
Expand Down