Skip to content

Conversation

armanbilge
Copy link
Contributor

Not sure if the precedent here is to avoid side-effecting APIs at all costs (if so, apologies), but personally I would find this useful. I started this PR to add an API for writing to an OutputStream but realized it could be more general.

@codecov-commenter
Copy link

codecov-commenter commented Nov 18, 2021

Codecov Report

Merging #1855 (b3a92c6) into main (4968a67) will decrease coverage by 0.19%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1855      +/-   ##
==========================================
- Coverage   86.21%   86.01%   -0.20%     
==========================================
  Files          73       73              
  Lines        2531     2531              
  Branches      143      143              
==========================================
- Hits         2182     2177       -5     
- Misses        349      354       +5     
Impacted Files Coverage Δ
.../core/shared/src/main/scala/io/circe/Printer.scala 94.25% <100.00%> (-0.96%) ⬇️
...re/shared/src/main/scala/io/circe/JsonNumber.scala 92.77% <0.00%> (-6.03%) ⬇️
.../core/shared/src/main/scala/io/circe/Decoder.scala 83.37% <0.00%> (-0.26%) ⬇️
...rc/main/scala/io/circe/numbers/BiggerDecimal.scala 88.30% <0.00%> (+1.75%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4968a67...b3a92c6. Read the comment docs.

Comment on lines 218 to 222
final def printToAppendable(json: Json, out: Appendable): Unit = {
val folder = new AppendableFolder(out)
json.foldWith(folder)
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR! I think circe has really tried to avoid public methods with side effecting signatures, so I'm fairly hesitant to introduce one. That being said, I think if we mark the boundaries of safeness and let others know that thar be dragons.. then we can probably let it in. So something like this:

Suggested change
final def printToAppendable(json: Json, out: Appendable): Unit = {
val folder = new AppendableFolder(out)
json.foldWith(folder)
}
object Unsafe {
final def printToAppendable(json: Json, out: Appendable): Unit = {
val folder = new AppendableFolder(out)
json.foldWith(folder)
}
}

And we should add some scaladoc to the method as well. For instance, it should be noted that this method returns only after all JSON is written to the appendable and that any use of the Appendable outside this method voids the warranty 😆 .

@armanbilge
Copy link
Contributor Author

I think circe has really tried to avoid public methods with side effecting signatures, so I'm fairly hesitant to introduce one.

Yes, that's completely fair! Thanks for being open to it. The goal here is mostly to expose a higher-performance API, so here's an idea to do it safely:

  1. Keep the changes I have here, but make it private[circe] def printToAppendable
  2. Then in circe-fs2, expose a public API that uses printToAppendable internally. This way we could use effects to provide a safe/pure API.

The catch with this is that even though printToAppendable is technically private to circe it would need the usual bincompat guarantees because circe-fs2 is released/versioned independently. This is easily mitigated with a comment drawing attention to this. What do you think?

Otherwise...

👍 to unsafe and docs. Just a suggestion: instead of an Unsafe object, what about naming the method unsafePrintToAppendable? This is how unsafe methods are named in Cats Effect, and has the advantage of putting unsafe directly at the call site (rather than a tricky import circe.Printer.Unsafe._ and then just an innocent-looking printToAppendable. No strong feelings either way, just a suggestion.

Thanks again for the review, and congrats on maintainership :)

@zarthross
Copy link
Member

zarthross commented Nov 24, 2021

I completely understand the goal of the higher-performance API.

Lets for now go with unsafePrintToAppendable and start work toward something in circe-fs2 to replace it. (We can always deprecate it before 1.0)

I'm thinking that we might need a different implementation completely instead of what we have here... maybe something based on Json.Folder[fs2.Stream] instead of the current side effectful Json.Folder[Unit].

Happy to help and thanks!

@armanbilge
Copy link
Contributor Author

Sounds good, updated!

I'm thinking that we might need a different implementation completely instead of what we have here...

I'm not sure. I was imagining a circe version of this fs2 method:

def writeOutputStream[F[_]](
    fos: F[OutputStream],
    closeAfterUse: Boolean = true
)(implicit F: Sync[F]): Pipe[F, Byte, INothing]

https://github.com/typelevel/fs2/blob/f0272ce00d930cf4eceaae67023517365c5fca6f/io/shared/src/main/scala/fs2/io/io.scala#L99-L108

So something like this (bikeshed the method names):

// this one prints a stream of JSON as an array:
// [{},{},{}]
def printArrayToAppendable[F[_]](
    fapp: F[Appendable],
)(implicit F: Sync[F]): Pipe[F, Json, INothing]

// this one uses a custom delimiter instead, e.g.:
// {}\n{}\n{}
def printToAppendable[F[_]](
    fapp: F[Appendable],
    delim: CharSequence,
)(implicit F: Sync[F]): Pipe[F, Json, INothing]

@zarthross
Copy link
Member

To bikeshed a bit, i was just imaging something of the signature def printToStream[F[_]](json: Json, cs: Charset): Stream[F, Byte] but we could go and do something like def printPipe[F[_]](cs: Charset): Pipe[F, Json, Byte] as well.

For your use case in Feral you would be able to then do printToStream(json).through(fs2.io.writeOutputStream(channel)) or something.

Reason I suggest a different printer implementation is I think having an FS2 native implementation would be able to suspend the encoding of the json better, and just overall work better with FS2 than trying to hack around Appendable

@zarthross zarthross requested a review from zmccoy November 24, 2021 16:24
@zarthross
Copy link
Member

@armanbilge Thanks again for the PR. I'll have @zmccoy give it a review, and we'll get this into the next release.

Copy link
Member

@zmccoy zmccoy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working through this with us! This looks good to me.

@zarthross zarthross merged commit 3c45802 into circe:main Jan 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants