-
Notifications
You must be signed in to change notification settings - Fork 58
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Data migrations with Julia functions on attributes #823
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks Kevin, this will be an awesome feature. First round of comments below.
345ea95
to
f7d1db6
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks Kevin! Here's a batch of small questions/comments. I'll pick up with more reviewing later but I wanted to get some stuff down now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK, with this round of comments I am through the code itself! Now I just need to go through the tests.
71a1afd
to
a2c432f
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Final round of comments on the tests.
Ready for re-review (!!!!!) I should note that there aren't any tests for aggregation and such unintended applications yet. |
…eton diagrams now work.
63e239d
to
be652bb
Compare
This PR upgrades the
@migration
macros with the ability to plug Julia code into appropriate locations. For instance:a
is an attribute of the target schema, one can writea=> x->body
, wherebody
may depend onx
and all the names in the source schema. This allows the produced migration to mutate the attributes in a more-or-less arbitrary way. The generality of this mutation is illustrated in the tests byTuringMachine
.h
is a hom in the target schema, we can writeh
's image in terms of an anonymous function whose value depends on all the names in the source schema, but one whose values will be combinatorial data rather than attribute values.a
of the target schema has as domain an object sent to a conjunctive diagram containing an objecty
, we may writea=> (b(y)|>(x->body))
to disambiguate the input.->
expressions can be replaced with blocks whose last line is such an expression.The behavior is basically that the macro builds a
DataMigration
superficially just as it used to, but where some values of thehom_map
of the underlying functor may beAttrExpr{:nothing}
, and those and perhaps other homs have their values modified upon actually migrated using the Julia functions stored in the migration'sparams
.The most immediate remaining concern is that the code relies on changes in lots of places, at all three of the key steps of the migration macros (parsing into values from
DiagrammaticPrograms.AST
, then into actualDiagram
s andDiagramHom
s, then doing the actual migration), while also allowing for several different syntaxes, and so is not so clean.The biggest large-scale concern is that we can't yet use multi-input Julia functions, as for instance you'd want to take a product of weighted graphs with the weights of a pair of edges given by the sum of the pair's member's weights. This is an actual semantic step beyond the model we've used so far, as the intermediate migration of something like
weight => (x,y) -> weight(x)+weight(y)
would need to sendWeight
toTuple{Weight,Weight}
or to@cases W:Weight; U:Weight
, and this splitting would need to be collapsed upon the actual migration. None of this shifting of attribute types happens with examples we can do up till now. TheTuple
solution seems easiest, but it does lead to the question of what happens if you have two different combinatorial types using the same attribute type a different number of times each. So I don't think there's an obvious choice of direction here.