Skip to content

Conversation

schomatis
Copy link
Contributor

@schomatis schomatis commented Jun 13, 2018

Modify fillNodeRec of the balanced importer to pass the UnixfsNode structure
instead of its pointer. The pointer was used as a channel to return information
from the recursive callee function which was obscuring the code information
flow. This returned information is now explicitly added to the return values of
fillNodeRec that now include the IPLD node to be added and its (UnixFS) file
size (that depends on the type of node and is handled by the encapsulating
UnixfsNode structure).

As a consequence of this, IPLD nodes can now be used instead of the composite
UnixfsNode in some parts of the code (which helps clarifying what type of node
is being manipulated in each case). Some of the functions that used UnixfsNode
as arguments were substituted with analogous functions that receive ipld.Node,
the original functions cannot be replaced (at the moment) because they are still
being used by the trickle builder.

The indirect modification of the passed UnixfsNode through its pointer was
used in the corner case of the Layout function when the loop is iterated only
once, i.e., the generated DAG is a single node with data and AddChildNode is
never called. In that case the root was converted through its pointer to a
leaf node with data in fillNodeRec which could later be added to the DAG
(db.Add()). This indirect modification is not allowed now so that corner case
needs to be handled separately by explicitly adding the filledRoot node
returned by fillNodeRec which wasn't added to root (instead of adding root
itself).

The priority of this commit is to respect the structure and code flow of the
importer to still make evident how a balanced DAG is built.

@schomatis schomatis added status/in-progress In progress topic/files Topic files labels Jun 13, 2018
@schomatis schomatis added this to the Files API Documentation milestone Jun 13, 2018
@schomatis schomatis self-assigned this Jun 13, 2018
@schomatis schomatis requested a review from Kubuxu as a code owner June 13, 2018 15:27
@schomatis schomatis removed the status/in-progress In progress label Jun 14, 2018
@schomatis schomatis changed the title [WIP] importer: fillNodeRec: pass UnixfsNode structure, not its pointer importer: fillNodeRec: pass UnixfsNode structure, not its pointer Jun 14, 2018
@schomatis
Copy link
Contributor Author

@whyrusleeping Could you please take a look at it? It addresses our discussion in #5106 (comment).

@ghost ghost added the status/in-progress In progress label Jun 14, 2018
Modify `fillNodeRec` of the balanced importer to pass the `UnixfsNode` structure
instead of its pointer. The pointer was used as a channel to return information
from the recursive callee function which was obscuring the code information
flow. This returned information is now explicitly added to the return values of
`fillNodeRec` that now include the IPLD node to be added and its (UnixFS) file
size (that depends on the type of node and is handled by the encapsulating
`UnixfsNode` structure).

As a consequence of this, IPLD nodes can now be used instead of the composite
`UnixfsNode` in some parts of the code (which helps clarifying what type of node
is being manipulated in each case). Some of the functions that used `UnixfsNode`
as arguments were substituted with analogous functions that receive `ipld.Node`,
the original functions cannot be replaced (at the moment) because they are still
being used by the trickle builder.

The indirect modification of the passed `UnixfsNode` through its pointer was
used in the corner case of the `Layout` function when the loop is iterated only
once, i.e., the generated DAG is a single node with data and `AddChildNode` is
never called. In that case the `root` was converted through its pointer to a
leaf node with data in `fillNodeRec` which could later be added to the DAG
(`db.Add()`). This indirect modification is not allowed now so that corner case
needs to be handled separately by explicitly adding the `filledRoot` node
returned by `fillNodeRec` which wasn't added to `root` (instead of adding `root`
itself).

The priority of this commit is to respect the structure and code flow of the
importer to still make evident how a balanced DAG is built.

License: MIT
Signed-off-by: Lucas Molas <schomatis@gmail.com>
@schomatis
Copy link
Contributor Author

There's no sense in working around the UnixfsNode, just remove it.

@schomatis schomatis closed this Jun 15, 2018
@schomatis schomatis deleted the feat/importer/fillrec-return branch June 15, 2018 01:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic/files Topic files
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant