Internally, it leverages a trick from Andy Gill's Kansas Lava to observe sharing in the tape it records for back propagation purposes, and uses type level branding to avoid confusing sensitivities.

I've tried to keep the API relatively close to that of Barak Pearlmutter and Jeffrey Mark Siskind's fad package, but I couldn't resist making a couple of minor tweaks here and there for generality.

I still need to go through and finish up the remaining unimplemented fad combinators, figure out a nice way to build a reverse-mode AD tower, validate that I didn't screw up my recollection of basic calculus, and provide a nice API for using this approach to get local reverse mode checkpoints in an otherwise forward mode AD program, but I am quite happy with how things have progressed thus far.

[Edit: I've uploaded minor bug fixes for exp and (**)]

The formula for classical precision is:

However, I would argue that the classical notion of Precision is flawed, in that it doesn't model anything we tend to care about. Rarely are we interested in binary classification, instead we want a ranked classification of relevance.

When Google tells you that you have a million results, do you care? No, you skim the first few entries for what it is that you are looking for, unless you are particularly desperate for an answer. So really, you want a metric that models the actual behavior of a search engine user and that level of desperation.

There are two issues with classical precision:

1. the denominator of precision goes to infinity as the result set increases in size
2. each result is worth the same amount no matter where it appears in the list

The former ensures that a million answers drowns out any value from the first screen, the latter ensures that it doesn't matter which results are on the first screen. A more accurate notion of precision suitable for modern search interfaces should model the prioritization of the results, and should allow for a long tail of crap if the stuff that people will look at is accurate over all.

So how to model user behavior? We can replace the denominator with a partial sum of a geometric series for probability p < 1, where p models the percentage chance that a user will continue to browse to the next item in the list. Then you can scale the value of the nth summand in the numerator as being worth up to pⁿ. If you have a ranked training set it is pretty easy to score precision in this fashion.

You retain all of the desirable properties of precision. It maxes out at 100%, it decreases when you give irrelevant results, but now it effectively models when you return irrelevant results early in your result list.

The result more accurately models user behavior when faced with a search engine than the classical binary precision metric. The parameter p models the desperation of the user and can vary to fit your problem domain. I personally like p=50%, because it makes for nice numbers, but it should proabably be chosen based on sampling based on knowledge of the search domain.

You can of course embellish this model with a stair-step in the cost function on each page boundary, etc. — any monotone decreasing infinite series that sums to a finite number in the limit should do.

A similar modification can of course be applied to recall.

I used this approach a couple of years ago to help tune a search engine to good effect. I went to refer someone to this post today and I realized I hadn't posted it in the almost two years since it was written, so here it is, warts and all.

If anyone is familiar with similar approaches in the literature, I'd be grateful for references!

The first talk covers an introduction to the mathematical notion of a monoid, introduces some of the features of my Haskell monoids library on hackage, and starts to motivate the use of monoidal parallel/incremental parsing, and the modification use of compression algorithms to recycle monoidal results.

The second talk covers a way to generate a locally-context sensitive parallel/incremental parser by modifying Iteratees to enable them to drive a Parsec 3 lexer, and then wrapping that in a monoid based on error productions in the grammar before recycling these techniques at a higher level to deal with parsing seemingly stateful structures, such as Haskell layout.

1. Introduction To Monoids (PDF)
2. Iteratees, Parsec and Monoids: A Parsing Trifecta (PDF)

Due to a late start, I was unable to give the second talk. However, I did give a quick run through to a few die-hards who stayed late and came to the Cambridge Brewing Company afterwards. As I promised some people that I would post the slides after the talk, here they are.

The current plan is to possibly give the second talk in full at either the September or October Boston Haskell User Group sessions, depending on scheduling and availability.

[ Iteratee.hs ]

• Hac Phi Slides (Powerpoint 2007)
• Hac Phi Slides (PDF)

There will be more to come at the next Boston Haskell User Group in August, where it looks like I'll be giving two short talks covering monoids. I may use the monoidal parsing engine from Kata as an example for the advanced talk if I have time and will start to cover parsing larger classes of grammars in general (regular languages, CFGs/TIGs, TAGs, PEGs, LALR, attribute-grammars, etc.)

The following recursion schemes can be found in category-extras, along with variations on the underlying themes, so this should work as a punch-list.

* This gives rise to a family of related recursion schemes, modeled in category-extras with distributive law combinators
† The scheme can be generalized to accept one or more F-distributive (co)monads.

Dinatural Transformations

Rather than repeat the definition here, we'll just note we can define a dinatural transformation in Haskell letting polymorphism represent the family of morphisms.

 
type Dinatural f g = forall a. f a a -> g a a
 

Ends

So what is an end?

An end is a universal dinatural transformation from some object e to some functor s.

Diving into the formal definition:

Given a functor , and end of is a pair where is an object of and omega is a dinatural transformation from e to S such that given any other dinatural transformation to S from another object x in , there exists a unique morphism , such that for every in .

We usually choose to write ends as , and abuse terminology calling the end of .

Note this uses a dinatural transformation from an object in , which we can choose to represent an arbitrary dinatural transformation from an object to a functor in terms of a dinatural transformation from the constant bifunctor:

 
newtype Const x a b = Const { runConst :: x }
 

This leaves us with the definition of a dinatural transformation from an object as:

 
Dinatural (Const x) s ~ forall a. Const x a a -> s a a
 

but the universal quantification over the Const term is rather useless and the Const bifunctor is supplying no information so we can just specialize that down to:

 
type DinaturalFromObject x s = x -> forall a. s a a
 

Now, clearly for any such transformation, we could rewrite it trivially using the definition:

 
type End s = forall a. s a a
 

with = id.

 
type DinaturalFromObject x s = x -> End s
 

And so End above fully complies with the definition for an end, and we just say e = End s abusing the terminology as earlier. The function = id is implicit.

For End to be a proper end, we assume that s is contravariant in its first argument and covariant in its second argument.

Example: Hom

A good example of this is (->) in Haskell, which is as category theory types would call it the Hom functor for Hask. Then End (->) = forall a. a -> a which has just one inhabitant id if you discard cheating inhabitants involving fix or undefined.

We write or to denote a -> b. Similarly we use to denote an exponential object within a category, and since in Haskell we have first class functions using the same syntax this also translates to a -> b. We'll need these later.

Example: Natural Transformations

If we define:

 
newtype HomFG f g a b =
        HomFG { runHomFG :: f a -> g b }
 

then we could of course choose to define natural transformations in terms of End as:

 
type Nat f g = End (HomFG f g) -- forall a. f a -> g a
 

Right Kan Extension as an End

Turning to Wikipedia or Categories for the Working Mathematician you can find the following definition for right Kan extension of along in terms of ends.

Working by rote, we note that is just c -> K m as noted above, and that is then just (c -> K m) -> T m'. So now we just have to take the end over that, and read off:

 
newtype RanT k t c m m' = (c -> k m) -> t m'
type Ran k t c = End (RanT k t c)
 

Which, modulo newtype noise is the same as the type previously supplied type:

 
newtype Ran f g c = Ran { runRan :: forall m. (c -> f m) -> g m }
 

Coends

The derivation for the left Kan extension follows similarly from defining coends over Hask in terms of existential quantification and copowers as products.

The coend derivation is complicated slightly by Haskell's semantics. Disposing of the constant bifunctor as before we get:

 
type DinaturalToObject s c = forall a. (s a a -> c)
 

Which since we want to be able to box up the s a a term separately, we need to use existential quantification.

 
(forall a. s a a -> c) ~ (exists a. s a a) -> c
 

We cannot represent this directly in terms of a type annotation in Haskell, but we can do so with a data type:

 
data Coend f = forall a. Coend (f a a)
 

Recall that in Haskell, existential quantification is represented by using universal quantification outside of the type constructor.

The main difference is that in our coend the function is now runCoend instead of id, because we have a Coend data constructor around the existential. Technicalities make it a little more complicated than that even, because you can't define a well-typed runCoend and have to use pattern matching on the Coend data constructor to avoid having an existentially quantified type escape the current scope, but the idea is the same.

Left Kan Extension as a Coend

Then given the definition for the left Kan extension of along as a coend:

we can read off:

 
data LanT k t c m m' = LanT (k m -> c) (t m')
type Lan k t c = Coend (LanT k t c)
 

Which is almost isomorphic to the previously supplied type:

 
data Lan k t c = forall m. Lan (k m -> c) (t m)
 

except for the fact that we had to use two layers of data declarations when using the separate Coend data type, so we introduced an extra place for a to hide.

A newtype isn't allowed to use existential quantification in Haskell, so this form forces a spurious case analysis that we'd prefer to do without. This motivates why category-extras uses a separate definition for Lan rather than the more elaborate definition in terms of Coend.

[Edit: Fixed a typo in the definition of RanT]

They appear to be black magic to Haskell programmers, but as Saunders Mac Lane said in Categories for the Working Mathematician:

All concepts are Kan extensions.

So what is a Kan extension? They come in two forms: right- and left- Kan extensions.

First I'll talk about right Kan extensions, since Haskell programmers have a better intuition for them.

Introducing Right Kan Extension

If we observe the type for a right Kan extension over the category of Haskell types:

 
newtype Ran g h a = Ran
        { runRan :: forall b. (a -> g b) -> h b }
 

This is defined in category-extras under Control.Functor.KanExtension along with a lot of the traditional machinery for working with them.

We say that Ran g h is the right Kan extension of h along g. and mathematicians denote it . It has a pretty diagram associated with it, but thats as deep as I'll let the category theory go.

This looks an awful lot like the type of a continuation monad transformer:

 
newtype ContT r m a = ContT
        { runContT :: (a -> m r) -> m r }
 

The main difference is that we have two functors involved and that the body of the Kan extension is universally quantified over the value it contains, so the function it carries can't just hand you back an m r it has lying around unless the functor it has closed over doesn't depend at all on the type r.

Interestingly we can define an instance of Functor for a right Kan extension without even knowing that g or h are functors! Anything of kind * -> * will do.

 
instance Functor (Ran g h) where
        fmap f m = Ran (\\k -> runRan m (k . f))
 

The monad generated by a functor

We can take the right Kan extension of a functor f along itself (this works for any functor in Haskell) and get what is known as the monad generated by f or the codensity monad of f:

 
instance Monad (Ran f f) where
	return x = Ran (\\k -> k x)
	m >>= k = Ran (\\c -> runRan m (\\a -> runRan (k a) c))
 

This monad is mentioned in passing in Opmonoidal Monads by Paddy McCrudden and dates back further to Ross Street's "The formal theory of monads" from 1972. The term codensity seems to date back at least to Dubuc's thesis in 1974.

Again, this monad doesn't care one whit about the fact that f is a Functor in the Haskell sense.

This monad provides a useful opportunity for optimization. For instance Janis Voigtländer noted in Asymptotic improvement of functions over Free Monads that a particular monad could be used to improve performance -- Free monads as you'll recall are the tool used in Wouter Sweirstra's Data Types á la Carte, and provide an approach for, among other things, decomposing the IO monad into something more modular, so this is by no means a purely academic exercise!

Voigtländer's monad,

 
newtype C m a = C (forall b. (a -> m b) -> m b)
 

turns out to be just the right Kan extension of another monad along itself, and can equivalently be thought of as a ContT that has been universally quantified over its result type.

The improvement results from the fact that the continuation passing style transformation it applies keeps you from traversing back and forth over the entire tree when performing substitution in the free monad.

The Yoneda Lemma

Heretofore we've only used right Kan extensions where we have extended a functor along itself. Lets change that:

Dan Piponi posted a bit about the Yoneda lemma a couple of years back, which ended with the observation that the Yoneda lemma says that check and uncheck are inverses:

 
> check :: Functor f => f a -> (forall b . (a -> b) -> f b)
> check a f = fmap f a
 
> uncheck :: (forall b . (a -> b) -> f b) -> f a
> uncheck t = t id
 

We can see that this definition for a right Kan extension just boxes up that universal quantifier in a newtype and that we could instantiate:

 
> type Yoneda = Ran Identity
 

and we can define check and uncheck as:

 
check' :: Functor f => f a -> Yoneda f a
check' a = Ran (\\f -> fmap (runIdentity . f) a)
 
uncheck' :: Yoneda f a -> f a
uncheck' t = runRan t Identity
 

Limits

We can go on and define categorical limits in terms of right Kan extensions using the Trivial functor that maps everything to a category with a single value and function. In Haskell, this is best expressed by:

 
data Trivial a = Trivial
instance Functor Trivial where
        fmap f _ = Trivial
trivialize :: a -> Trivial b
trivialize _ = Trivial
 
type Lim = Ran Trivial
 

Now, in Haskell, this gives us a clear operational understanding of categorical limits.

 
Lim f a ~ forall b. (a -> Trivial b) -> f b
 

This says that we can't use any information of the value a we supply, or given by the function (a -> Trivial b) when constructing f b, but we have to be able to define an f b for any type b requested. However, we have no way to get any b to plug into the functor! So the only (non-cheating) member of Lim Maybe a is Nothing, of Lim [] a is [], etc.

Left Kan extensions

Left Kan extensions are a bit more obscure to a Haskell programmer, because where right Kan extensions relate to the well-known ContT monad transformer, the left Kan extension is related to a less well known comonad transformer.

First, the a Haskell type for the Left Kan extension of h along g:

 
data Lan g h a = forall b. Lan (g b -> a) (h b)
 

This is related to the admittedly somewhat obscure state-in-context comonad transformer, which I constructed for category-extras.

 
newtype ContextT s w a = ContextT
        { runContextT :: (w s -> a, w s) }
 

However, the left Kan extension provides no information about the type b contained inside of its h functor and g and h are not necessarily the same functor.

As before we get that Lan g h is a Functor regardless of what g and h are, because we only have to map over the right hand side of the contained function:

 
instance Functor (Lan f g) where
	fmap f (Lan g h) = Lan (f . g) h
 

The comonad generated by a functor

We can also see that the left Kan extension of any functor f along itself is a comonad, even if f is not a Haskell Functor. This is of course known as the comonad generated by f, or the density comonad of f.

 
instance Comonad (Lan f f) where
	extract (Lan f a) = f a
	duplicate (Lan f ws) = Lan (Lan f) ws
 

Colimits

Finally we can derive colimits, by:

 
type Colim = Lan Trivial
 

then Colim f a ~ exists b. (Trivial b -> a, f b), and we can see that operationally, we have an f of some unknown type b and for all intents and purposes a value of type a since we can generate a Trivial b from thin air, so while limits allow only structures without values, colimits allow arbitrary structures, but keep you from inspecting the values in them by existential quantification. So for instance you could apply a length function to a Colim [] a, but not add up its values.

You can also build up a covariant analog of the traditional Yoneda lemma using Lan Identity, but I leave that as an exercise for the reader.

I've barely scratched the surface of what you can do with Kan extensions, but I just wanted to shine a little light on this dark corner of category theory.

For more information feel free to explore category-extras. For instance, both right and left Kan extensions along a functor are higher-order functors, and hence so are Yoneda, Lim, and Colim as defined above.

Thats all I have time for now.

Code for right and left Kan extensions, limits, colimits and the Yoneda lemma are all available from category-extras on hackage.

[Edit: the code has since been refactored to treat Yoneda, CoYoneda, Density and Codensity as separate newtypes to allow for instance both Yoneda and Codensity to be different monads]

I want to talk about a novel recursion scheme that hasn't received a lot of attention from the Haskell community and its even more obscure dual -- which is necessarily more obscure because I believe this is the first time anyone has talked about it.

Jiri Adámek, Stefan Milius and Jiri Velebil have done a lot of work on Elgot algebras. Here I'd like to translate them into Haskell, dualize them, observe that the dual can encode primitive recursion, and provide some observations.

You can kind of think an Elgot algebra as a hylomorphism that cheats.

 
> elgot :: Functor f => (f b -> b) -> (a -> Either b (f a)) -> a -> b
> elgot phi psi = h where h = (id ||| phi . fmap h) . psi
 

If you look at the signature for a hylomorphism:

 
> hylo :: Functor f => (f b -> b) -> (a -> f a) -> a -> b
> hylo phi psi = h where h = phi . fmap h . psi
 

Then you can see that an Elgot algebra is basically a hylomorphism that is allowed to shortcircuit the infinite tower of fmaps and return an intermediate result directly.

In some sense you can say that the coalgebra-like side of the hylomorphism is no longer oblivious to the algebra used to deconstruct the intermediate result.

We can take the Elgot algebra and dualize it to get a novel construction where the algebra-like side is no longer oblivious to the coalgebra. This allows your algebra to cheat and just use the intermediate results constructed by the anamorphism to return an answer. I'll choose to call this co-(Elgot algebra) an Elgot coalgebra in the sequel.

 
> coelgot :: Functor f => ((a, f b) -> b) -> (a -> f a) -> a -> b
> coelgot phi psi = h where h = phi . (id &&& fmap h . psi)
 

In a lot of ways an Elgot algebra resembles Vene and Uustalu's apomorphism.

 
> apo :: Functor f => (a -> f (Either (Mu f) a)) -> a -> Mu f
> apo psi = h where h = InF . fmap h . (fmap Left . outF ||| psi) . Right
 

However, we have 'unfixed' the algebra to be used from InF to something more general and the layering of Either and f is different.

Now, a generalized apomorphism does something similar entangling two coalgebras, but the signature doesn't quite match up either, since a generalized apomorphism uses an F-coalgebras and an F-(b + _)-monadic coalgebra.

 
> g_apo :: Functor f => (b -> f b) -> (a -> f (Either b a)) -> a -> Mu f
> g_apo g f = h where h = InF . fmap h . (fmap Left . g ||| f) . Right
 

Similarly a zygomorphism, or more generally a mutumorphism entangles two algebras.

An Elgot algebra occupies a somewhat rare spot in the theory of constructive algorithmics or recursion schemes in that it while it mixes an algebra with a coalgebra like a hylomorphism or metamorphisms, it entangles them in a novel way.

If we specialize the Elgot algebra by fixing its algebra to InF we get:

 
> elgot_apo :: Functor f => (a -> Either (Mu f) (f a)) -> a -> Mu f
> elgot_apo psi = h where h = (id ||| InF . fmap h) . psi
 

We can see that the type is now closely related to that of an apomorphism with some slight changes in design decisions. Instead of wrapping a functor around further seeds, a, or a finished structure, this specialized Elgot algebra returns the finished structure directly or an f wrapped around seeds.

The Good

So can we convert between an apomorphism and an Elgot algebra? For a somewhat circuitous path to that answer lets recall the definition of strength from my post a couple of weeks ago. Flipping the arguments and direction of application for strength to simplify what is coming we get:

 
> strength' :: Functor f => t -> f a -> f (t, a)
> strength' fa b = fmap ((,)b) fa
 

With that in hand we quickly find that we can rederive paramorphisms (and hence primitive recursion) from the novel notion of an Elgot coalgebra that we defined above by leaning on the strength of our functor.

 
> para :: Functor f => (f (Mu f, c) -> c) -> Mu f -> c
> para f = coelgot (f . uncurry strength') outF
 

This result tells us that the shiny new Elgot coalgebras we defined above are strong enough to encode primitive recursion when working in Haskell.

The Bad

This tempts us to try to derive apomorphisms from Elgot algebras using the dual case, costrength. However, if you'll recall from my previous post on comonadic costrength, we can't do that in general. The result is only defined for Traversable functors; not every functor is costrong in Haskell!

Consequently and counterintuitively, though we can define a paramorphism in terms of Elgot coalgebras, we can only define an apomorphism in terms of Elgot algebras for traversable functors.

The Ugly

Now, worse news. Since the tower of functors we build up doesn't run off to infinity we lose the ability to generalize Elgot (co)algebras using the same machinery we can use to generalize the various traditional recursion schemes by parameterizing it by a (co)monad and distributive law.

At least the straightforward translation fails. For instance in the case of an Elgot algebra, the obvious addition would be to allow for the algebra (f a -> a) to be replaced with a F-W-comonadic algebra (f (w a) -> a) for some comonad w. However, attempts to do so run afoul of the fact that the coalgebra-like structure feeds us an 'a' not a 'w a'. We can of course change the signature of the coalgebra to give us the comonad, but the breakdown of modularity is unfortunate.

Similary, parameterizing the coalgebra-like structure with a monad requires the ability to distribute the monad over Either b to get to where it can apply the distributive law for the base functor f. Interestingly the Either monad works, which gives us ways to compose Elgot (co)algebras, but that is a story for another day.

As usual there is a tradeoff in expressivity in one area to compensate for gains in another, but this manner of entangling provides us with a new set of possibilities to explore.

Code for Elgot algebras and Elgot coalgebras has been included in category-extras as of release 0.50.3 as Control.Functor.Algebra.Elgot.

Now available from hackage.

I dusted off my copy of Herbert Wilf's generatingfunctionology and switched goals to try to see some well known recursive functors or species as formal power series. It appears that we can pick a few things out about the generating functions of polynomial functors.

As an example:

 
Maybe x = 1 + x
 

Ok. We're done. Thank you very much. I'll be here all week. Try the veal...

For a more serious example, the formal power series for the list [x] is just a geometric series:

 
[x] = mu y . 1 + x y  -- the mu here is a pleasant fiction, more below
    = 1 + x (1 + x (1 + x (...)))
    = 1 + x + x^2 + x^3 + ...
    = 1/(1-x)
 

Given the power series of a functor, its nth coefficient * n! tells you how many distinguishable ways its constructors can house n values. If we see that a list of n values can be permuted n! ways this has some interesting possibilities for linearizing the storage of some functors. The list case is boring, we can store a finite list of n elements by supplying the length of the array and an array of n elements, hence (among other reasons) the mu above.

Lets try decorated binary trees:

 
data Tree x = Leaf | Node (Tree x) x (Tree x)
 
Tree x = mu y. 1 + x * y * y
       = 1 + x * (1 + x * (...)^2)^2
       = 1 + x + 2x^2 + 5x^3 + 14x^4 + 42x^5 + ...
 

It turns out the coefficients of our generating function are the Catalan numbers, A000108, commonly denoted C(n), which happen to be well known for among other things, being the number of ways you can build a binary tree of n nodes.

This tells us we could store a tree by storing a number n of nodes it contains, an array of that many nodes, and an index 0 < = i < C(n) to tell you which particular tree you selected. Not that this is likely to be an incredibly time-efficient encoding, but you could then fmap over the tree by just fmapping over your array.

For a formal power series,

its derivative is given by differentiating the series term by term:

Consequently we can take the derivative of a list:

([]') x = 1 + 2x + 3x^2 + ... = 1/(1-x)^2 = ([] :*: []) x

and rederive the notion that a derivative/one hole context of a list can be represented by a pair of lists.

If we step slightly outside of the Haskell users' comfort zone and notion of a Functor and allow other Species, we get (as noted by apfelmus the other day) that Exp a is just a bag of elements with no predetermined order.

 
Exp x = 1 + x + x^2/2! + x^3/3! + ... = Bag x
 
Since there are n! ways to order n elements and Bag manages to forget that information, we can get a feeling for the meaning of division in this setting.
 
Similarly we can define:
<pre lang="haskell">
Sinh x = x + x^3/3! + ... -- a Bag of some odd number of elements.
Cosh x = 1 + x^2/2! + ... --  a Bag of some even number of elements.
 

Then by construction:

 
Exp = Cosh :+: Sinh
 

The derivative of Exp is Exp, of Cosh is Sinh, of Sinh is Cosh, all as you would expect.

We can handle other species as well:

 
Cycle a = 1 + x^2/2 + x^3/3 + x^4/4 + ... -- cycles
Cycle_n a = x^n/n -- cycles of n elements
Bag_n a = x^n/n!  -- bags of n elements
 

That said, there seem to be some problems, not every functor is well behaved in this way. Lets take for instance the type of natural numbers given by the functor:

 
data Nat a = S (Nat a) | Z
 

Then the recurrence blows up, the coefficient for 0 is !

Similarly, if we parameterized a functor on another value we have to deal with the number of cases that other value can denote.

 
Either a x = |a| + x
(a,x) = |a| x
 

This is both good and bad, using the above, we can quickly establish an isomorphism between Either () and the Maybe Functor, but we blow up again for Either Integer. This gets even worse if we allow 'functors in space.' (i.e. functors that can contain functions)

On the other extreme, we might modify our tree example and remove the leaves, yielding infinite decorated cotrees.

 
Tree x = nu y. x * y * y
       = x * (x * (...)^2)^2
       = 0 + 0x + 0x^2 + 0x^3 + ...
 

Then a_n = 0 for all n in the natural numbers, so you can't use the coefficients of the generating function to tell you about the behavior of an infinite structure! It would appear that the generating function of a functor does not capture what happens in the greatest fixed point case, so we can only use generating functions to describe the behavior of data defined with mu, not in general codata defined by nu.

The Bags and Cycles above are nice examples, but if we wanted to rule out the non-polynomial Functors (from the Haskell perspective) in the above then we can simply limit ourselves to ordinary generating functions with natural number coefficients, that is to say generating functions of the form:

To choose to admit bags, cycles and other species etc. then you need merely also permit exponential generating functions with natural coefficients, that is to say, generating functions of the form:

There has been a lot of posting recently about automatic differentiation in Haskell, and I wanted to try the same thing with functors in the spirit of Conor McBride's Clowns to the Left of me, Jokers to the Right and The derivative of a regular type is its type of one hole contexts, figuring that a Power Series could fully generalize Christophe Poucet's Higher Order Zippers, and might provide me with a neat extension to the zipper comonadic automata I've been aluding to recently.

 
{-# OPTIONS -fglasgow-exts -fallow-undecidable-instances -fallow-overlapping-instances #-}
module Derivatives where
import Control.Monad.Identity
import Control.Arrow ((+++),(***),(&&&))
import Data.Monoid
infixl 9 :.:
infixl 7 :*:
infixl 6 :+:
 

To avoid importing category-extras and keep this post self-contained (modulo GHC 6.9!), we'll define some preliminaries such as Bifunctors:

 
class Bifunctor f where
	bimap :: (a -> c) -> (b -> d) -> f a b -> f c d
 
instance Bifunctor (,) where
	bimap f g ~(a,b) = (f a, g b)
 
instance Bifunctor Either where
	bimap f _ (Left a) = Left (f a)
	bimap _ g (Right b) = Right (g b)
 

Constant functors:

 
data Void
instance Show Void where show _ = "Void"
newtype Const k a = Const { runConst :: k } deriving (Show)
type Zero = Const Void
type One = Const ()
instance Functor (Const k) where
	fmap f = Const . runConst
 

and functor products and coproducts:

 
newtype Lift p f g a = Lift { runLift ::  p (f a) (g a) }
type (:+:) = Lift Either
type (:*:) = Lift (,)
instance Show (p (f a) (g a)) => Show (Lift p f g a) where
	show (Lift x) = "(Lift (" ++ show x ++ "))"
instance (Bifunctor p, Functor f, Functor g) => Functor (Lift p f g) where
	fmap f = Lift . bimap (fmap f) (fmap f) . runLift
 

and finally functor composition

 
newtype (f :.: g) a = Comp { runComp :: f (g a) } deriving (Show)
instance (Functor f, Functor g) => Functor (f :.: g) where
	fmap f = Comp . fmap (fmap f) . runComp
 

So then, an ideal type for repeated differentiation would look something like the following, for some definition of D.

[Edit: sigfpe pointed out, quite rightly, that this is just repeated differentiation, and apfelmus pointed out that it not a power series, because I have no division!]

 
newtype AD f a  = AD { runAD :: (f a,  AD (D f) a) }
 

As a first crack at D, you might be tempted to just go with a type family:

 
{-
type family D (f :: * -> *) :: * -> *
type instance D Identity = One
type instance D (Const k) = Zero
type instance D (f :+: g) = D f :+: D g
type instance D (f :*: g) = f :*: D g :+: D f :*: g
type instance D (f :.: g) = (D f :.: g) :*: D g
-}
 

This could take you pretty far, but unfortunately doesn't adequately provide you with any constraints on the type so that we can treat AD f as a functor.

So, we'll go with:

 
class (Functor (D f), Functor f) => Derivable (f :: * -> *) where
	type D f :: * -> *
 

and cherry pick the instances necessary to handle the above cases:

 
instance Derivable Identity where
	type D Identity = One
 
instance Derivable (Const k) where
	type D (Const k) = Zero
 
instance (Derivable f, Derivable g) => Derivable (f :+: g) where
	type D (f :+: g) = D f :+: D g 
 
instance (Derivable f, Derivable g) => Derivable (f :*: g) where
	type D (f :*: g) = f :*: D g :+: D f :*: g
 
instance (Derivable f, Derivable g) => Derivable (f :.: g) where
	type D (f :.: g) = (D f :.: g) :*: D g
 

With those instances in hand, we can define the definition of a Functor for the automatic differentiation of a Functor built out of these primitives:

 
instance (Derivable f, Functor (AD (D f))) => Functor (AD f) where
	fmap f = Power . bimap (fmap f) (fmap f) . runPower
 

Unfortunately, here is where I run out of steam, because any attempt to actually use the construct in question blows the context stack because the recursion for Functor (AD f) isn't well founded and my attempts to force it to be so through overlapping-instances have thus-far failed.

Thoughts?

[Source Code]