ephemeral404@lemmy.mlM to Data Engineering@lemmy.mlEnglish · 3 years agoData Engineering roadmaplemmy.mlimagemessage-square4linkfedilinkarrow-up17arrow-down10file-text
arrow-up17arrow-down1imageData Engineering roadmaplemmy.mlephemeral404@lemmy.mlM to Data Engineering@lemmy.mlEnglish · 3 years agomessage-square4linkfedilinkfile-text
minus-squareAndy@lemmy.mllinkfedilinkEnglisharrow-up1·3 years agoIsn’t Apache Arrow a table format rather than a batch processing tool? You could add SQLMesh as an alternative to dbt.
minus-squareephemeral404@lemmy.mlOPMlinkfedilinkEnglisharrow-up1·3 years agothanks for suggesting sqlmesh. regarding arrow, have you read this - https://arrow.apache.org/docs/format/Columnar.html
Isn’t Apache Arrow a table format rather than a batch processing tool?
You could add SQLMesh as an alternative to dbt.
thanks for suggesting sqlmesh. regarding arrow, have you read this - https://arrow.apache.org/docs/format/Columnar.html