Repository navigation
Conversation
|
I'll read in details later but I just have a quick comment so far. I don't think we should return an error when the rewrite is impossible. The idea is that rewrite will be identity when nothing can be improved, so that users (or even us) can call rewrite without a priori knowledge of the rewritability of the tensor, and let cten check that for them. |
|
Cool, I checked what Rust allows:
Should I proceed as a whole or just do 1? |
Is there any issue with that? Seems like the proper way of implementing this Maybe we can switch to specialization once it becomes more stable. |
Shall we wait for it to be stable ? |
|
Could take a long time. What's the issue with the runtime-based compilation? |
|
There's no blocker, it's mostly more work. The main points:
I can try and do it if it's okay with u? |
Sorry for the late reply. We're basically looking into intermediate representation simplification. Maybe a graph structure would help us, but for now I don't think it's needed. Maybe such a representation could be helpful internally, but honestly it's quite independent from this PR and from the problem we currently have. It's just an idea. |
|
Done, pushed it. rewrite now takes any tensor and returns an Since f̄ ∘ Ū = (f ∘ U)‾, it rewrites f ∘ Ω ∘ Ū ∘ ψ as Ω ∘ f̄ ∘ Ū ∘ ψ, and that works for any reduction as long as f keeps its neutral element (Lemma 6), Theorem 1 being the sum case. In the tests, squaring a sparse 2x4 matrix with 3 stored values calls the function 3 times after rewrite instead of 8. |
rewrite(tensor), which turns a tensor into an equivalent one that is cheaper to evaluate, or gives it back unchanged asErrwhen no identity applies.f ∘ Σ ∘ Ū ∘ ψ = Σ ∘ (f ∘ U)‾ ∘ ψ, so a zero-preserving function like square or relu is only applied to the stored values. It checks that f(0) = 0 and that the tensor is sparse.Sparsitytrait so that index functions (COO, Diagonal, EmptyMapping, InvProjection, InvAffine) can tell whether they send each index to at most one index.