[SYSTEMDS-3857] set get names on dataframes - #2495
Conversation
This patch adds the language references for the newly implemented getName and setName function. The order in Builtins.java was fixed to be alphabetical again
t99-i
changed the title
janniklinde
left a comment
janniklinde
left a comment
There was a problem hiding this comment.
Thanks for the PR @t99-i. I left some comments in the code that should be addressed.
In general, please configure your IDE to use tabs instead of spaces. Also, please remove the unnecessary TODO comments.
As a next step, please implement support for the spark backend and add the according tests. In general, you should have a more systematic approach to verifying metatdata propagation through tests as you currently only include cbind/rbind/slice for the CP backend (should be extended to spark backend and should test other frame related functions systematically).
github-project-automation
Bot
moved this from In Progress
to In Review
in SystemDS PR Queue
- fixed mapping of binarOP in DMLTranslator - added size/data validation in BinaryFrameFrameCPInstruction - setName does now have a STRING return type - removed duplicated code - fixed get/set-swap - removed unnecessary prints in FrameColumnNamesTest - removed unnecessary TODOs
- added SetName functionality for SPARK - extended propagation test (wip) - added Set/GetNames function tests for SPARK
- Adapted BuiltinNarySPInstruction to carry over columnNames on FrameBlocks - Removed SetColumnNamesFunction from FrameAppendRSPInstruction - Adapted MLContextConversionUtil to carry over columnNames on FrameBlocks - Adapted ParameterizedBuiltinFEDInstruction to carry over schema and columnNames on FrameBlocks - Adapted ParameterizedBuiltinFEDInstruction to carry over schema and columnNames on FrameBlocks
- replaced the setColumnMetadata function in FrameBlock with a deep copy variant
- small fix in the FrameColNamesPropagationTest
- added dml file for left Indexing test
- added a readme for the student project
…et-names-on-dataframes # Conflicts: # src/main/java/org/apache/sysds/runtime/controlprogram/caching/FrameObject.java
- removed unused function
|
Your changes don't build @t99-i. Could you please fix these errors to allow a more detailed evaluation of the code? Thanks. |
- added class for SetColumnNamesFunction with helper function for frmabelock handling
Oh yeah i see it, i forgot to track a file, im really sorry, it should build now |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #2495 +/- ##
============================================
+ Coverage 71.61% 71.64% +0.03%
- Complexity 50352 50511 +159
============================================
Files 1626 1629 +3
Lines 194664 195140 +476
Branches 38007 38068 +61
============================================
+ Hits 139408 139814 +406
- Misses 44328 44382 +54
- Partials 10928 10944 +16 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
2 participants
This PR extends the handling of frame column names by introducing support for retrieving and updating column names in DML, improving metadata propagation across frame operations, and fixing a remaining metadata propagation edge case.
Changes:
Motivation:
Previously, frame column names could not be modified through DML and were not consistently preserved across frame operations, particularly for distributed frame representations. This PR provides a consistent runtime representation for column name metadata while maintaining backward compatibility.
Tests:
Added unit tests for getNames() and setNames().
Added metadata propagation tests for relevant frame operations.
Added regression tests for the one-off propagation bug.
Verified existing frame functionality remains unchanged.