This is a live mirror of the Perl 5 development currently hosted at https://github.com/perl/perl5
Adjust callsites that use UTF8SKIP without checking UTF
authorAlex Vandiver <alex@chmrr.net>
Mon, 23 Mar 2015 03:44:11 +0000 (23:44 -0400)
committerDavid Mitchell <davem@iabyn.com>
Mon, 29 Feb 2016 09:44:27 +0000 (09:44 +0000)
Assuming UTF-8 semantics and advancing character-by-character when 'use
utf8' is not enabled is not as problematic as the inverse.  However,
properly UTF8SKIP should only be used when UTF8 semantics are explicitly
asked for.

Change the three occurrences of UTF8SKIP that are not protected by UTF
checks.

toke.c

diff --git a/toke.c b/toke.c
index c3d3c97..e5ef340 100644 (file)
--- a/toke.c
+++ b/toke.c
@@ -5856,12 +5856,12 @@ Perl_yylex(pTHX)
                    else
                        /* skip plain q word */
                        while (t < PL_bufend && isWORDCHAR_lazy_if(t,UTF))
-                            t += UTF8SKIP(t);
+                           t += UTF ? UTF8SKIP(t) : 1;
                }
                else if (isWORDCHAR_lazy_if(t,UTF)) {
-                   t += UTF8SKIP(t);
+                   t += UTF ? UTF8SKIP(t) : 1;
                    while (t < PL_bufend && isWORDCHAR_lazy_if(t,UTF))
-                        t += UTF8SKIP(t);
+                       t += UTF ? UTF8SKIP(t) : 1;
                }
                while (t < PL_bufend && isSPACE(*t))
                    t++;