Injecting implicits into functions

[fancellu]

I have a function

 def withTxn[T](f: Context=>T)={
  implicit val txn=Context(true)
  val ret=f(txn)
  txn.commit()
  ret
}

To use it I have to invoke thus

withTxn{implicit txn=>
 val user = makeUser(id="user_test1234",username="test1234")
}

// makeUser takes an implicit Context

I'd like to be able to invoke like this

withTxn{
 val user = makeUser(id="user_test1234",username="test1234")
 }

i.e. it "injects" the implicits invisibly. Less characters, less noise.

[propensive]

I've thought about this idea a few times in the past (I think I proposed something similar on the Scala mailing list in about 2007...), as it would offer a lot of interesting possibilities.

But a problem with it is that the closure parameter to your withTxn implicitly introduces a new scope, which may trade the benefits you describe for more surprise and confusion when someone isn't expecting this to be the case.

Now, this isn't so different from the new scope you get when you create a new nested (possibly anonymous) class, e.g. (this is a contrived example -- I wish I could have found a better one in the standard library):

  object Foo {
    def available = "This is a string"
    val x = new scala.concurrent.Lock {
      println(available.substring(5))
    }
  }

This won't compile because the available introduced by the new Lock scope shadows the one defined outside, which we might naively assume we're referring to.

So the hurdle we would need to jump would be educating people that not only can a new class introduce a new scope, but an ordinary closure can too, and therefore that every closure could be introducing a new scope.
Unfortunately, I think that could introduce too much uncertainly into writing Scala code. I'm not certain about this, but it seems like it could encourage people to introduce too much magic.

(But experimentation is exactly what Typelevel Scala is for!)

An alternative solution is to take advantage of exactly what I've described above and to change your API to expect an anonymous instance of a class which provides the implicits, instead of the implicit lambda, e.g.

  // definitions
  class Txn {
    implicit def txn = ...
  }
  def withTxn(blk: Txn) = ...

  // usage
  withTxn { new Txn {
    val user = makeUser(id="user_test1234", username="test1234")
  } }

Though there are plenty of reasons why this would be unsatisfactory.

[fancellu]

Well, if they don't know what withTxn does, shouldn't use it! Just need it to be clear, in IDE etc, that it is present, i.e. this function has these N implicits (i.e. could even be more than 1)

Any time I keep repeating myself in code I get a bad feeling.

withTxn injects a Context into f(), I'm just trying to get rid of some extra chars, where the compiler injects "implicit txn=>" in there

[propensive]

Yeah, I understand and completely agree with the motivation! We just have to be cautious that powerful features can get overused, and degrade the whole coding experience by introducing uncertainty, whether that was the original intention or not.

This said, it is already possible to implement what you want using macros. The basic strategy would be to create dummy implicits of the right types in global scope, just so that the lambda will typecheck, and then to rewrite the lambda in the macro to replace all references to these dummy implicits with references to the lambda parameter(s).

I don't want to discourage this because I think it could be a useful feature, though there would be some reluctance from me to have it become part of Typelevel Scala. But that's not to say I couldn't be convinced after playing around with it for a few months -- it's absolutely a feature I would use.

[stanch]

The most recent discussion: https://groups.google.com/d/topic/scala-language/Ow-OPG2E76E/discussion. The one before it: https://groups.google.com/d/topic/scala-debate/f4CLmYShX6Q/discussion.

[propensive]

They're the same link! But the discussion they refer to about DSL Paradise
is very interesting:

https://github.com/dsl-paradise/dsl-paradise

On 13 September 2014 05:15, Nick [email protected] wrote:

The most recent discussion:
https://groups.google.com/d/topic/scala-language/Ow-OPG2E76E/discussion.
The one before it:
https://groups.google.com/d/topic/scala-language/Ow-OPG2E76E/discussion

—
Reply to this email directly or view it on GitHub
#44 (comment).

Jon Pretty | @propensive

[stanch]

@propensive should be fixed by now. dsl-paradise is more of a statement than something actually working, but @lihaoyi has done a terrific job describing various use-cases.

[lihaoyi]

DSL paradise is actually two separate (but related!) ideas:

• Injecting a type-based implicit into a function's parameter's expression
• Injecting a bunch of names into a function's parameter's expression

Apart from the motivation as described on the dsl-paradise github, here's two contemporary examples where the idea came up completely un-prompted:

• A recent thread on scala-language, which re-created the first idea
• paulp's twitter, where he basically described the second idea to the letter

I think the fact that people come up with the idea over and over and over on their own is worth some weight =)

The first, as you described (and I didn't realize), can entirely be implemented with clever macro munging with dummy @compileTimeOnly implicits, so I guess that use case is taken care of =) The second would need untyped macros to implement as-a-library, and is probably a bigger endeavor overall.

[fancellu]

If someone has done the first way with Macros, it would be nice to see the code. I tried and due to the current lack of IDE support for Macro dev, I didn't have much luck. (lots of huge long compile errors, or simply not working)

Of course the second case, done properly, would be even nicer. As soon as you get boiler plate code, again and again, you know its a bad sign. Its so easy to just accept it, rationalize it, or get used to it, hence Java.

[stanch]

There is at least one caveat to the macro implementation: if the implicits contain path-dependent types, it may not be possible to unify several call sites with the same global implicit.

[ScalaWilliam]

+1. I keep on seeing this pattern. Maybe there is a different approach to solving this problem.

Example:
withTransaction {implicit entityManager => implicit entityTransaction => ... }

Would be awesome to see a nice way to simplify this pattern. I hate repeating myself in code.

[paulp]

I think a language built around first class scopes would be wonderful. But that language is never going to be scala. And I think scala with a bunch of ad hoc rules of this kind would be significantly less wonderful.

[maffstephens]

I'm working with the "withTxn" code in the example, and (typing too fast) just managed to miss out the "implicit" in "implicit txn". Caught it shortly afterwards, but it's just the sort of boilerplate avoidance I'd hope a language like Scala would help encapsulate. Wouldn't it just be an extra implicit resolution rule?

[som-snytt]

Because what's one more implicit resolution rule?

With sufficient tooling to bestow x-ray vision when the machinery breaks down, it shouldn't matter what the rules are.

We just watched the third episode of Superman with George Reeves. He peers inside the safe (about to fall from block-and-tackle to the street below) and sees Jimmy Olson inside. Later he looks inside a safe and discovers the stolen money. Does it matter whether the safe contains cash or Jimmy Olson? All that matters is that we catch the bad guys. The safe is just a black box.

[Blaisorblade]

In the original idea, is the idea that withTxn would become

def withTxn[T](f: (implicit Context)=>T)={

? I ask because if so, this feature is available in Agda (that does have better-thought-out implicit support in general, see #8). Even there, there are tricky rules governing this. I might be able to find their documentation or a paper with a good, non-ad-hoc formulation; even then, this might be too confusing.

[fancellu]

Yes, probably could be defined something like that. I just want a programming language where I can enforce contracts to avoid boilerplate and errors in the boilerplate, flips sides of the same coin.

[propensive]

@Blaisorblade I think either that, or

def withTxn[T](f: (Context @implicit) => T) = ...

Scala already has a binary encoding for type annotations, so whichever syntax we use, we'd probably want to encode implicitness as a type annotation.

I'm not sure what this would mean for binary compatibility at the call site, but what we would need is for

withTxn { implicit ctx => ... }

to compile in both Typesafe and Typelevel Scala, whereas

withTxn { ... }

would only work in Typelevel Scala.

[Blaisorblade]

I think that's a great idea, and can be applied even for #8 — we had figured we needed to extend binary signatures somehow.

An annoying detail is that parameter lists can be implicit, but annotations are for arguments. The solution is of course to add the annotation to a fixed parameter (probably the first), but it's inelegant, and in other contexts I go to greater lengths against smaller inelegances.

However, that seems like a reason to not use annotations in the source syntax: at least, we (I?) don't want (@implicit foo, bar) to mean that both params are implicit, because the inelegant transformation should be completely hidden.

[aaronlifton3]

This doesn't make sense. You can already do this with implicit parameters.

def makeUser(name: String)(implicit db: DbConn) = ...
def withDB(x: Unit) {
  implicit val db = new DbConn(...)
  x()
}

withDB {
  makeUser("Bob")
}

[propensive]

That doesn't work! The implicit db is not in scope inside the parameter to withDB.

[stanch]

@aaronlifton Have you actually tried running that code? If yes, can I haz your compiler? Also Unit does not have an apply method.

Proof:

scala> :paste
// Entering paste mode (ctrl-D to finish)

case class DbConn()

def makeUser(name: String)(implicit db: DbConn) = ???
def withDb(x: Unit) {
  implicit val db = DbConn()
  x
}

withDb {
  makeUser("Bob")
}

// Exiting paste mode, now interpreting.

<console>:18: error: could not find implicit value for parameter db: DbConn
                makeUser("Bob")
                        ^

[aaronlifton3]

Sorry. Using the Reader Monad you can achieve this.

 object Reader {

    /**
     *  automatically wrap a function in a reader
     */
    implicit def reader[From, To](block: From => To) = Reader[From, To](block)

    /**
     * create a reader that does not really depend on anything. This is a way of wrapping an already computed result (like a failure) within a Reader when we need such a return type.
     */
    def pure[From, To](a: To) = Reader((c: From) => a)

    /**
     * resolve a reader
     */
    def withDependency[F, T](dep: F)(reader: Reader[F, T]): T = reader(dep)

    def withConn[F, T](reader: Reader[F, T])(implicit dep: F): T = reader(dep)

  }

  /**
   * The reader Monad
   */
  case class Reader[-From, +To](wrappedF: From => To) {

    def apply(c: From) = wrappedF(c)
    // f(c)

    def map[ToB](transformF: To => ToB): Reader[From, ToB] =
      Reader(c => transformF(wrappedF(c)))
      // f(g(c)

    def flatMap[FromB <: From, ToB](f: To => Reader[FromB, ToB]): Reader[FromB, ToB] =
      Reader(c => f(wrappedF(c))(c))
      // f(g(c)).f(c)
      // map().apply()
  }
//Model
  case class User(id: Long)
  case class Post(title: String)

  //Connections
  trait UserConnection {

    def readUser(id: Long): Option[User] = Some(User(id))

  }

  trait PostConnection {

    def readPosts(user: User): Seq[Post] = Seq(Post("test"), Post("test2"))

  }

  import Reader._

  //a concrete connection
  type UserPostConn = UserConnection with PostConnection
  class RealConn extends UserConnection with PostConnection {}
  val conn = new RealConn

  /**
   * all the posts from a user please
   */
  def userPosts(userID: Long): Reader[UserPostConn, Seq[Post]] = reader { conn =>
    conn.readUser(userID) map { user =>
      conn.readPosts(user)
    } getOrElse List() //getting rid of the Option just to simplify the code in this article
  }

  /**
   * just the titles, thank you
   */
  def titles(id: Long) = userPosts(id).map { postIterable =>
    postIterable.map(_.title)
  }

  /**
   * unwrap the reader given a concrete dependency
   */
  withDependency(conn) { titles(10l) }

trait HasConnection {
    // import Reader._
    type UserPostConn = UserConnection with PostConnection
    class RealConn extends UserConnection with PostConnection {}
    implicit val conn: UserPostConn = new RealConn
  }


  object TitleController extends HasConnection {
    def getTitles = titles(1)
    def run = {
      withDependency(conn) {
        getTitles
      }
      // OR
      withConn {
        getTitles
      }
    }

Note the last line

[stanch]

Of course. And it takes almost exactly the same amount of code as

withDb { implicit conn ⇒
  makeUser("Bob")
}

Except that now you have to wrap everything inside withDb into a Reader, which makes it less simple and less performant...

[milessabin]

To resurrect this issue, please rework it as an issue/PR against Lightbend Scala (ie. scala/scala).