Discussion:
[Mason-devel] extending <% | %>
Hans Dieter Pearcey
2008-01-16 16:12:04 UTC
Permalink
Attached is a patch that makes | in substitutions more powerful. This isn't
finished; it does work, and all but two of the existing tests pass with it, but
the implementation feels a bit fragile and there's no documentation.

There are two major changes.

The first is that there can be multiple |s in a substitution, and each can have
arguments (which can be arbitrary Perl). I called these 'filters', even though
that's kind of an overloaded word now (so I'm open to suggestions).

The second is that instead of evaluating the LHS of a substitution immediately,
it's wrapped in sub { } and passed to the filters that way; this gives more
control to the filters, because evaluation is delayed.

Some examples will probably be clearer:

<% $object->expensive_method(\%args) | cache(expires_in => "10m") | h %>

(there's a test that demonstrates that in t/10-cache.t)

<% $biller->calculate_amount | format('$%5.2f') %>

The old escape_flags and apply_escapes are now implemented in terms of filters.
They keep working anywhere in the filter pipeline; the only special casing done
for them is that escape flags are parsed specially when they're the last filter
entry, so that this keeps working:

<% $foo | h, u %>

But this would also work:

<% $foo | h | u | cache %>

I also did some refactoring of cache_self -- '| cache(...)' uses the same
arguments as cache_self, and I didn't want to reinvent the wheel. See
HTML::Mason::Request::_cache_access (the name sucks). We probably want
something like this independent of the rest of the patch; it makes writing
cache_self analogues much easier.

Because LHS evaluation is delayed, I had to do some ugly things to make sure
that globals referred to on the LHS were still around at $m->print time,
especially $_ and @_.

One thing I can't fix is that wantarray becomes useless in <% %> (it will
always be true). It used to reflect the calling context of exec().

Look at HTML::Mason::Filters for how cache() and format() are written. In
particular, the latter is written pretty much exactly how an 'escape' used to
be.

Things I have not yet done, but consider necessary or very useful for this
being 'finished':

* cache() should autogenerate a unique cache key per <% %> (and maybe even
per-filter, per-substitution), so that it's trivial to stick '| cache' onto
existing substitutions

* more real-world useful filters

* documentation and tests, obviously

If all goes well, I would next move on to something like this:

<%| cache %>
A bunch of text here, including <% $more->stuff %>
with maybe some <& /component/calls &>
you know, lots of text, that would be annoying to put inside a
<% $substitution %>
</%>

(but let's paint that bikeshed after we're done with this one)

hdp.
Hans Dieter Pearcey
2008-01-16 16:16:02 UTC
Permalink
Post by Hans Dieter Pearcey
Attached is a patch that makes | in substitutions more powerful.
And, of course, it isn't. Oops.

hdp.
Jonathan Swartz
2008-01-16 20:38:49 UTC
Permalink
Wow, thanks Hans! We've talked for a while about reconciling |,
filters, and components-with-content. Looking forward to taking a
look at your patch.

Jon
Post by Hans Dieter Pearcey
Attached is a patch that makes | in substitutions more powerful.
This isn't
finished; it does work, and all but two of the existing tests pass with it, but
the implementation feels a bit fragile and there's no documentation.
There are two major changes.
The first is that there can be multiple |s in a substitution, and each can have
arguments (which can be arbitrary Perl). I called these 'filters', even though
that's kind of an overloaded word now (so I'm open to suggestions).
The second is that instead of evaluating the LHS of a substitution immediately,
it's wrapped in sub { } and passed to the filters that way; this gives more
control to the filters, because evaluation is delayed.
<% $object->expensive_method(\%args) | cache(expires_in => "10m") | h %>
(there's a test that demonstrates that in t/10-cache.t)
<% $biller->calculate_amount | format('$%5.2f') %>
The old escape_flags and apply_escapes are now implemented in terms of filters.
They keep working anywhere in the filter pipeline; the only special casing done
for them is that escape flags are parsed specially when they're the last filter
<% $foo | h, u %>
<% $foo | h | u | cache %>
I also did some refactoring of cache_self -- '| cache(...)' uses the same
arguments as cache_self, and I didn't want to reinvent the wheel. See
HTML::Mason::Request::_cache_access (the name sucks). We probably want
something like this independent of the rest of the patch; it makes writing
cache_self analogues much easier.
Because LHS evaluation is delayed, I had to do some ugly things to make sure
that globals referred to on the LHS were still around at $m->print time,
One thing I can't fix is that wantarray becomes useless in <% %> (it will
always be true). It used to reflect the calling context of exec().
Look at HTML::Mason::Filters for how cache() and format() are
written. In
particular, the latter is written pretty much exactly how an
'escape' used to
be.
Things I have not yet done, but consider necessary or very useful for this
* cache() should autogenerate a unique cache key per <% %> (and maybe even
per-filter, per-substitution), so that it's trivial to stick '| cache' onto
existing substitutions
* more real-world useful filters
* documentation and tests, obviously
<%| cache %>
A bunch of text here, including <% $more->stuff %>
with maybe some <& /component/calls &>
you know, lots of text, that would be annoying to put inside a
<% $substitution %>
</%>
(but let's paint that bikeshed after we're done with this one)
hdp.
----------------------------------------------------------------------
---
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Mason-devel mailing list
https://lists.sourceforge.net/lists/listinfo/mason-devel
Otto Hirr
2008-01-27 22:41:54 UTC
Permalink
The first is that there can be multiple |s in a substitution, ...
I called these 'filters', even though that's kind of an overloaded
word now (so I'm open to suggestions).
Something jogged my mind back to a book I scanned a while ago...
_Standard C++ IOStreams and Locales_ by Langer and Kreft

There seems to be, maybe in my warped sense, an analogy between
what is going on in mason and iostreams. The have a diagram:

program
,--^-----------------,
| |formatting layer | iostreams
| |transport layer |
|__v_________________/
ext dev

which has filters, traits, etc.

Maybe some of the arch and terminology is appropriate. Maybe
some of the abstractions that have developed for iostreams,
may be a design patterns appropriate for next-gen mason.
No re-invention of wheel.

Am I way off on a tangent?

..Otto
Hans Dieter Pearcey
2008-01-31 10:46:32 UTC
Permalink
Post by Hans Dieter Pearcey
Attached is a patch that makes | in substitutions more powerful.
The biggest problem is with using '|'. While I think it's unlikely that anyone
is doing lots of bitwise-or inside <% %>, it's not impossible that someone
would write a regex that matches the pattern I'm using for finding filters; it
is roughly:

/\| \s* \w+(\(.*\))? \s* $/x

which matches, for example,

<% $x =~ m(foo|bar(baz)?) %>

I can't think of a really satisfactory way to solve this. I definitely do not
want to get too heavily into trying to parse Perl, and the only alternative I
can think of is to use a different delimiter for these kinds of filters.

I generated a whole bunch of examples with different delimiters, and I've
attached them, but I don't really like any of them, either, and most of them
wouldn't completely sidestep the problem anyway.

Anyone have better ideas? I would love to hear them.

hdp.
Scott Lanning
2008-02-14 12:41:30 UTC
Permalink
Post by Hans Dieter Pearcey
Post by Hans Dieter Pearcey
Attached is a patch that makes | in substitutions more powerful.
The biggest problem is with using '|'. While I think it's unlikely that anyone
is doing lots of bitwise-or inside <% %>, it's not impossible that someone
would write a regex that matches the pattern I'm using for finding filters; it
/\| \s* \w+(\(.*\))? \s* $/x
which matches, for example,
<% $x =~ m(foo|bar(baz)?) %>
I can't think of a really satisfactory way to solve this. I definitely do not
want to get too heavily into trying to parse Perl, and the only alternative I
can think of is to use a different delimiter for these kinds of filters.
I generated a whole bunch of examples with different delimiters, and I've
attached them, but I don't really like any of them, either, and most of them
wouldn't completely sidestep the problem anyway.
Anyone have better ideas? I would love to hear them.
Would backslashing non-delimiting pipes be bad?

<% $x =~ m(foo\|bar(baz)?) %>


The list of delimiters that you made
made me think that pipes could probably be generalized
to filehandles or something -- I guess that's the same
as what Otto was getting at with streams.
You could probably easily get carried away, though.
Like I was thinking how you could pipe SQL to a database
(there's already an implicit redirection of the output
of <% %> to "out_method"), get its output, pass to whatever
formatter... but in the end, it probably would be just as easy
to do it in a <%perl> block. Note for example that something like

<% $object->expensive_method(\%args) | cache(expires_in => "10m") %>

could also be done from a <%perl> block, leaving

<% $cached_val %>

within the non-code part of the template.
It has to fit with whatever model <% %> is meant to have,
not just be implemented because it can be. IMHO the model
for <% %> can change, but I think currently it's mainly
intended/used for outputting a variable or small expression
within a larger body of non-code, like for a quick change
of context (like how % at the beginning of a line lets
you insert a single line instead of using a block).
Hans Dieter Pearcey
2008-02-14 15:12:54 UTC
Permalink
Post by Scott Lanning
Would backslashing non-delimiting pipes be bad?
<% $x =~ m(foo\|bar(baz)?) %>
Well, it breaks backcompat, which is one of the things I was hoping to avoid.
Post by Scott Lanning
Like I was thinking how you could pipe SQL to a database
I admit that this is a pretty interesting idea, but probably not what we want
to encourage people to do in their templates. :)
Post by Scott Lanning
Note for example that something like
<% $object->expensive_method(\%args) | cache(expires_in => "10m") %>
could also be done from a <%perl> block, leaving
<% $cached_val %>
within the non-code part of the template.
Of course it could. Responding to a sugar proposal by saying "you could do
this without the sugar" misses the point, though.

First, there's the fact that it's currently very inconvenient to do the "get
from cache or set", because it's not abstracted away in a generic way. That
doesn't require new syntax, though; something like this could wrap it all up:

$m->cached_or_do(
sub { $obj->calc_some_stuff(@args) },
expires_in => "10m",
cache_key => "your face",
# ... other cache options ...
);

Once we have that, the real difference is between these two:

-----

<%perl>
my $cached_val = $m->cached_or_do(
sub { $obj->calc_some_stuff(@args) },
expires_in => "10m",
cache_key => "your face",
);
</%perl>

# ... somewhere else ...
<% $cached_val |h %>

-----

<% $obj->calc_some_stuff(@args)
| cache(expires_in => "10m", cache_key => "your face")
| h %>

-----

I prefer the second:
* the code you care about is at the top
* 'modifiers' (cache, h) are clearly separated
* maintained in one place in the code, not two

I keep coming back to 'cache' because I think it's the biggest potential win.
Fine-grained caching like this is something that I'm not aware of in any other
Perl template module; for example, Template::Plugin::Cache works at the
template level, not at the individual [% statement %] level.

Caching for even a short period of time can have a huge effect on application
responsiveness, so we should make it as easy as possible for people to cache
output. HTML and URI escaping got special sugar because people need to use
them all the time. If caching is far behind, I think it's because it's
frequently too much hassle, not because it wouldn't be useful.

hdp.
BenRifkah Bergsten-Buret
2008-02-15 20:37:55 UTC
Permalink
Post by Scott Lanning
-----
<%perl>
my $cached_val = $m->cached_or_do(
expires_in => "10m",
cache_key => "your face",
);
</%perl>
# ... somewhere else ...
<% $cached_val |h %>
-----
| cache(expires_in => "10m", cache_key => "your face")
| h %>
+1 for being able to send parameters to custom escapes.

However, I'm not sold that using the pipe to chain escapes is really
necessary. I agree that the pipe symbol is great because it already
carries the desired meaning because its used in numerous other
contexts. However, implementing it seems to me to be more trouble than
its worth especially since the comma is already implemented (and used in
the wild) for chaining escapes.

I think that using pipes as the delimiter for chaining escapes may be
another bike shed to paint in the future :)

-- Ben

P.S.: I actually like the first syntax because IMO it allows for more
effective separation of storage and display, but to each his own.
Hans Dieter Pearcey
2008-02-15 21:38:08 UTC
Permalink
Post by BenRifkah Bergsten-Buret
However, I'm not sold that using the pipe to chain escapes is really
necessary. I agree that the pipe symbol is great because it already
carries the desired meaning because its used in numerous other
contexts. However, implementing it seems to me to be more trouble than
its worth especially since the comma is already implemented (and used in
the wild) for chaining escapes.
The problem is not having multiple pipes -- the problem is extending the syntax
of the pipe RHS. As soon as you allow '| foo(<arbitrary perl>)', the pipe is
a problem, without any chaining entering the picture.
Post by BenRifkah Bergsten-Buret
P.S.: I actually like the first syntax because IMO it allows for more
effective separation of storage and display, but to each his own.
I was arguing for both being available; I focused on the merits of the inline
syntax because it's the new feature I want to add.

hdp.
BenRifkah Bergsten-Buret
2008-02-15 23:21:24 UTC
Permalink
Post by Hans Dieter Pearcey
Post by BenRifkah Bergsten-Buret
However, I'm not sold that using the pipe to chain escapes is really
necessary. I agree that the pipe symbol is great because it already
carries the desired meaning because its used in numerous other
contexts. However, implementing it seems to me to be more trouble than
its worth especially since the comma is already implemented (and used in
the wild) for chaining escapes.
The problem is not having multiple pipes -- the problem is extending the syntax
of the pipe RHS. As soon as you allow '| foo(<arbitrary perl>)', the pipe is
a problem, without any chaining entering the picture.
Ah. Now I see the dilemma. I was stuck on vertical bars on the LHS of
the pipe. It's not a huge problem if you only have to account for pipes
on the LHS, but if you allow arbitrary perl (including vertical bars) on
the RHS then parsing can become a lot more difficult.

If you only have to account for '| foo(<arbitrary perl>)' then you might
be able to get some help from Text::Balanced. Allowing for '|
foo(<arbitrary perl>) | bar(<arbitrary perl>)' would be more difficult
but doable. (Not that you implied it but if you have to allow '|
<arbitrary perl>' I think you're going to be SOL.)

While I agree that it would be nice to allow for this arbitrary perl I
think that tackling it iteratively would be a good idea.

-- Ben
Hans Dieter Pearcey
2008-02-16 01:01:05 UTC
Permalink
Post by BenRifkah Bergsten-Buret
Ah. Now I see the dilemma. I was stuck on vertical bars on the LHS of
the pipe. It's not a huge problem if you only have to account for pipes
on the LHS, but if you allow arbitrary perl (including vertical bars) on
the RHS then parsing can become a lot more difficult.
If you only have to account for '| foo(<arbitrary perl>)' then you might
be able to get some help from Text::Balanced. Allowing for '|
foo(<arbitrary perl>) | bar(<arbitrary perl>)' would be more difficult
but doable. (Not that you implied it but if you have to allow '|
<arbitrary perl>' I think you're going to be SOL.)
I had thought of Text::Balanced and ended up discarding it; I think this was
because I hadn't considered disallowing '|' in filter arguments, or something.
Either that or there was some problem with using it because I had to start at
the end of the string (I know, reverse). I can't remember.

If I have time this weekend I'll look at it again.

hdp.

Loading...