%% This is part of the OpTeX project, see http://petr.olsak.net/optex

\_codedecl \begtt {Verbatim <2022-04-23>} % preloaded in format

   \_doc ----------------------------
   The internal parameters
   \`\_ttskip`, \`\_ttpenalty`, \`\_viline`, \`\_vifile` and \`\_ttfont`
   for verbatim macros are set.
   \_cod ----------------------------

\_def\_ttskip{\_medskip}           % space above and below \begtt, \verbinput
\_mathchardef\_ttpenalty=100       % penalty between lines in \begtt, \verbinput
\_newcount\_viline                 % last line number in \verbinput
\_newread\_vifile                  % file given by \verinput
\_def\_ttfont{\_tt}                % default tt font

   \_doc ----------------------------
   \`\code``{<text>}` expands to `\detokenize{<text>}` when `\escapechar=-1`. In
   order to do it more robust when it is used in `\write` then it expands as
   noexpanded `\code<space>` (followed by space in its csname). This macro
   does the real work.

   The \`\_printinverbatim``{<text>}` macro is used for `\code{<text>}` printing and for
   \code{`}<text>\code{`} printing. It is defined as `\hbox`, so the in-verbatim <text>
   will be never broken. But you can re-define this macro.

   When `\code` occurs in PDF outlines then it does the same as `\detokenize`.
   The macro for preparing outlines sets `\escapechar` to $-1$ and uses
   \^`\_regoul` token list before `\edef`.

   The `\code` is not `\proteced` because we want it expands to
   `\unexpanded{\code<space>{<text>}}` in `\write` parameters. This protect the
   expansions of the `\code` parameter (like `\\`, `\^` etc.).
   \_cod ----------------------------

\_def\_code#1{\_unexpanded\_ea{\_csname _code \_endcsname{#1}}}
\_protected\_sdef{_code }#1{{\_escapechar=-1 \_ttfont \_the\_everyintt \_relax
   \_ea\_printinverbatim\_ea{\_detokenize{#1}}}}
\_def\_printinverbatim#1{\_leavevmode\_hbox{#1}}

\_regmacro {}{}{\_let\code=\_detokenize \_let\_code=\_detokenize}
\_public \code ;

   \_doc ----------------------------
   The \`\_setverb` macro sets all catcodes to \"verbatim mode". It should be used only
   in a group, so we prepare a new catcode table with \"verbatim" catcodes and we define
   it as\nl `\_catcodetable`\`\_verbatimcatcodes`. After the group is finished then
   original catcode table is restored.
   \_cod ----------------------------

\_newcatcodetable \_verbatimcatcodes
\_def\_setverb{\_begingroup
   \_def\do##1{\_catcode`##1=12 }
   \_dospecials
   \_savecatcodetable\_verbatimcatcodes % all characters are normal
   \_endgroup
}
\_setverb
\_def\_setverb{\_catcodetable\_verbatimcatcodes }%

   \_doc ----------------------------
   \`\verbchar``<char>` saves original catcode of previously declared `<char>` (if
   such character was declared) using \`\_savedttchar` and \`\_savedttcharc`
   values. Then new such values are stored. The declared character is activated
   by `\_adef` as a macro (active character) which opens a group,
   does `\_setverb` and other settings and reads its parameter until second the same
   character. This is done by the \`\_readverb` macro. Finally, it prints scanned
   `<text>` by \^`\_printinverbatim` and closes group. Suppose that `\verbchar"` is
   used. Then the following work is schematically done:
   \begtt
   \_def "{\_begingroup \_setverb ... \_readverb}
   \_def \_readverb #1"{\_printinverbatim{#1}\_endgroup}
   \endtt
   Note that the second occurrence of `"` is not active because `\_setverb`
   deactivates it.
   \_cod ----------------------------

\_def\_verbchar#1{%
   \_ifx\_savedttchar\_undefined\_else \_catcode\_savedttchar=\_savedttcharc \_fi
   \_chardef\_savedttchar=`#1
   \_chardef\_savedttcharc=\_catcode`#1
   \_adef{#1}{\_begingroup \_setverb \_adef{ }{\_dsp}\_ttfont \_the\_everyintt\_relax \_readverb}%
   \_def\_readverb ##1#1{\_printinverbatim{##1}\_endgroup}%
}
\_let \_activettchar=\_verbchar % for backward compatibility
\_public \verbchar \activettchar ;

   \_doc ----------------------------
   \`\begtt` is defined only as public. We don't need a private `\_begtt` variant.
   This macro opens a group and sets `%` as an active character (temporary).
   This will allow it to be used as the comment character at the same line after
   `\begtt`. Then \`\_begtti` is run. It is defined by \^`\eoldef`,
   so users can put a parameter at the
   same line where `\begtt` is. This `#1` parameter is used after \^`\everytt`
   parameters settings, so users can change them locally.

   The \^`\_begtti` macro does \^`\_setverb` and another preprocessing, sets
   `\endlinechar` to `^^J` and reads the following text in verbatim mode
   until \`\endtt` occurs. This scanning is done by \`\_startverb` macro which is
   defined as:
   \begtt \adef/{\bslash}
   \_def\_startverb #1/endtt #2^^J{...}
   \endtt
   We must to ensure that the backslash in `\endtt` has category 12 (this is a
   reason of the `\ea` chain in real code).
   The `#2` is something between `\endtt` and the end of the same line and it is simply
   ignored.

   The `\_startverb` puts the scanned data to \`\_prepareverbdata`. It sets the data
   to `\_tmpb` without changes by default, but you should re-define it in order
   to do special changes if you want. (For example, \^`\hisyntax` redefines
   this macro.) The scanned data have `^^J` at each end of line and all spaces are
   active characters (defined as {\visiblesp`\ `}).
   Other characters have normal category 11 or 12.

   The `^^J` is appended to verbatim data because we need to be sure that
   the data are finished by this character. When `\endtt` is preceded by
   spaces then we need to close these spaces by `^^J` and such line is not
   printed due to a trick used in \^`\_printverb`.

   When `\_prepareverbdata` finishes then `\_startverb` runs `\_printverb` loop
   over each line of the data and does a final work: last skip plus `\noindent`
   in the next paragraph.
   \_cod ---------------------------

\_def\begtt{\_par \_begingroup \_adef\%##1\_relax{\_relax}\_begtti}
\_eoldef \_begtti#1{\_wipeepar \_setxhsize
   \_vskip\_parskip \_ttskip
   \_setverb
   \_ifnum\_ttline<0 \_let\_printverblinenum=\_relax \_else \_initverblinenum \_fi
   \_adef{ }{\_dsp}\_adef\^^I{\t}\_parindent=\_ttindent \_parskip=0pt
   \_def\t{\_hskip \_dimexpr\_tabspaces em/2\_relax}%
   \_protrudechars=0 % disable protrusion
   \_the\_everytt \_relax #1\_relax \_ttfont
   \_def\_testcommentchars##1\_iftrue{\_iffalse}\_let\_hicomments=\_relax
   \_savemathsb \_endlinechar=`^^J
   \_startverb
}
\_ea\_def\_ea\_startverb \_ea#\_ea1\_csstring\\endtt#2^^J{%
   \_prepareverbdata\_tmpb{#1^^J}%
   \_ea\_printverb \_tmpb\_fin
   \_par \_restoremathsb
   \_endgroup \_ttskip
   \_isnextchar\_par{}{\_noindent}%
}
\_def\_prepareverbdata#1#2{\_def#1{#2}}

   \_doc
   The \`\_printverb` macro calls \^`\_printverbline``{<line>}` repeatedly
   to each scanned line of verbatim text. The `\_printverb` is used from
   `\begtt...\endtt` and from `\verbinput` too.

   The \^`\_testcommentchars` replaces the following `\_iftrue` to
   `\_iffalse` by default unless the \~`\commentchars` are set. So, the main body
   of the loop is written in the `\_else` part of the `\_iftrue` condition.
   The \^`\_printverbline``{<line>}` is called here.

   The \`\_printverbline``{<line>}` expects that it starts in vertical mode and it must
   do `\par` to return the vertical mode. The \`\_printverblinenum`
   is used here: it does nothing when `\_ttline`\code{<0} else it prints the line
   number using `\_llap`.

   \`\_putttpenalty` puts \^`\_ttpenalty` before second and next lines, but
   not before first line in each `\begtt...\endtt` environment.

   The `\_ttline` is increased here in the `\_printverb` macro because of
   comments-blocks: the `\_prinverbline` is not processed in comments-blocks
   but we need to count the `\_ttline`.
   \_cod ----------------------------

\_def\_printverb #1^^J#2{%
   \_ifx\_printverblinenum\_relax \_else \_incr\_ttline \_fi
   \_testcommentchars #1\_relax\_relax\_relax
   \_iftrue
      \_ifx\_fin#2\_printcomments\_fi
   \_else
      \_ifx\_vcomments\_empty\_else  \_printcomments \_def\_vcomments{}\_fi
      \_ifx\_fin#2%
         \_bgroup \_adef{ }{}\_def\t{}% if the last line is emtpy, we don't print it
         \_ifcat&#1&\_egroup \_ifx\_printverblinenum\_relax \_else \_decr\_ttline \_fi
         \_else\_egroup \_printverbline{#1}\_fi
      \_else
         \_printverbline{#1}%
      \_fi
   \_fi
   \_unless\_ifx\_fin#2\_afterfi{\_printverb#2}\_fi
}
\_def\_printverbline#1{\_putttpenalty \_indent \_printverblinenum \_kern\_ttshift #1\_par}
\_def\_initverblinenum{\_tenrm \_thefontscale[700]\_ea\_let\_ea\_sevenrm\_the\_font}
\_def\_printverblinenum{\_llap{\_sevenrm \_the\_ttline\_kern.9em}}
\_def\_putttpenalty{\_def\_putttpenalty{\_penalty\_ttpenalty}}

   \_doc ----------------------------
   Macro \`\verbinput` uses a file read previously or opens the given file. Then
   it runs the parameter scanning by \`\_viscanparameter` and \`\_viscanminus`.
   Finally the \`\_doverbinput` is run. At the beginning of `\_doverbinput`, we have
   `\_viline`= number of lines already read using previous `\verbinput`,
   `\_vinolines`= the number of lines we need to skip and `\_vidolnes`= the
   number of lines we need to print.
   A similar preparation is done as in `\begtt` after the group is opened. Then
   we skip \`\_vinolines` lines in a loop a and we read \`\_vidolines` lines. The
   read data is accumulated into `\_tmpb` macro. The next steps are equal to
   the steps done in \^`\_startverb` macro: data are processed via
   \^`\_prepareverbdata` and printed via \^`\_printverb` loop.
   \_cod \_fin ----------------------

\_def\_verbinput #1(#2) #3 {\_par \_def\_tmpa{#3}%
   \_def\_tmpb{#1}%  cmds used in local group
   \_ifx\_vifilename\_tmpa \_else
      \_openin\_vifile={#3}%
      \_global\_viline=0 \_glet\_vifilename=\_tmpa
      \_ifeof\_vifile
         \_opwarning{\_string\verbinput: file "#3" unable to read}
         \_ea\_ea\_ea\_skiptorelax
      \_fi
   \_fi
   \_viscanparameter #2+\_relax
}
\_def\_skiptorelax#1\_relax{}

\_def \_viscanparameter #1+#2\_relax{%
   \_if$#2$\_viscanminus(#1)\_else \_viscanplus(#1+#2)\_fi
}
\_def\_viscanplus(#1+#2+){%
   \_if$#1$\_tmpnum=\_viline
   \_else \_ifnum#1<0 \_tmpnum=\_viline \_advance\_tmpnum by-#1
       \_else \_tmpnum=#1
             \_advance\_tmpnum by-1
             \_ifnum\_tmpnum<0 \_tmpnum=0 \_fi % (0+13) = (1+13)
   \_fi \_fi
   \_edef\_vinolines{\_the\_tmpnum}%
   \_if$#2$\_def\_vidolines{0}\_else\_edef\_vidolines{#2}\_fi
   \_doverbinput
}
\_def\_viscanminus(#1-#2){%
   \_if$#1$\_tmpnum=0
      \_else \_tmpnum=#1 \_advance\_tmpnum by-1 \_fi
   \_ifnum\_tmpnum<0 \_tmpnum=0 \_fi  % (0-13) = (1-13)
   \_edef\_vinolines{\_the\_tmpnum}%
   \_if$#2$\_tmpnum=0
      \_else \_tmpnum=#2 \_advance\_tmpnum by-\_vinolines \_fi
   \_edef\_vidolines{\_the\_tmpnum}%
   \_doverbinput
}
\_def\_doverbinput{%
   \_tmpnum=\_vinolines
   \_advance\_tmpnum by-\_viline
   \_ifnum\_tmpnum<0
      \_openin\_vifile={\_vifilename}%
      \_global\_viline=0
   \_else
      \_edef\_vinolines{\_the\_tmpnum}%
   \_fi
   \_vskip\_parskip \_ttskip \_wipeepar \_setxhsize
   \_begingroup
   \_ifnum\_ttline<-1 \_let\_printverblinenum=\_relax \_else \_initverblinenum \_fi
   \_setverb \_adef{ }{\_dsp}\_adef\^^I{\t}\_parindent=\_ttindent \_parskip=0pt
   \_def\t{\_hskip \_dimexpr\_tabspaces em/2\_relax}%
   \_protrudechars=0 % disable protrusion
   \_the\_everytt\_relax \_tmpb\_relax \_ttfont
   \_savemathsb \_endlinechar=`^^J \_tmpnum=0
   \_loop \_ifeof\_vifile \_tmpnum=\_vinolines\_space \_fi
         \_ifnum\_tmpnum<\_vinolines\_space
         \_vireadline \_advance\_tmpnum by1 \_repeat      %% skip lines
   \_edef\_ttlinesave{\_global\_ttline=\_the\_ttline}%
   \_ifnum\_ttline=-1 \_ttline=\_viline \_else \_let\_ttlinesave=\_relax \_fi
   \_tmpnum=0 \_def\_tmpb{}%
   \_ifnum\_vidolines=0 \_tmpnum=-1 \_fi
   \_ifeof\_vifile \_tmpnum=\_vidolines\_space \_fi
   \_loop \_ifnum\_tmpnum<\_vidolines\_space
            \_vireadline
            \_ifnum\_vidolines=0 \_else\_advance\_tmpnum by1 \_fi
            \_ifeof\_vifile \_tmpnum=\_vidolines\_space \_else \_visaveline \_fi %% save line
            \_repeat
   \_ea\_prepareverbdata \_ea \_tmpb\_ea{\_tmpb^^J}%
   \_catcode`\ =10 \_catcode`\%=9 % used in \commentchars comments
   \_ea\_printverb \_tmpb\_fin
   \_ttlinesave
   \_par \_restoremathsb
   \_endgroup
   \_ttskip
   \_isnextchar\_par{}{\_noindent}%
}
\_def\_vireadline{\_read\_vifile to \_tmp \_incr\_viline }
\_def\_visaveline{\_ea\_addto\_ea\_tmpb\_ea{\_tmp}}

\_public \verbinput ;

   \_doc -----------------------------
   \`\_savemathsb`, \`\_restoremathsb` pair is used
   in \^`\begtt`...\^`\endtt` or in \^`\verbinput` to temporary suppress
   the \^`\mathsbon` because we don't need to print `\int _a`
   in verbatim mode if `\int``_a` is really written. The \^`\_restoremathsb`
   is defined locally as \^`\mathsbon` only if it is needed.
   \_cod -----------------------------

\_def\_savemathsb{\_ifmathsb \_mathsboff \_def\_restoremathsb{\_mathsbon}\_fi}
\_def\_restoremathsb{}

   \_doc -----------------------------
   If the language of your code\label[commentchars]\wlabel{}
   printed by \^`\verbinput` supports the
   format of comments started by two characters from the beginning of the
   line then you can set these characters by \^`\commentchars``<first><second>`.
   Such comments are printed in the non-verbatim mode without these two
   characters and they look like the verbatim printing is interrupted
   at the places where such comments are.
   See the section~\ref[lua] for good illustration.
   The file `optex.lua` is read by a single command
   `\verbinput (4-) optex.lua` here and the `\commentchars --` was set before it.

   If you need to set a special character by \^`\commentchars` then you must
   to set the catcode to 12 (and space to 13). Examples:
   \begtt
   \commentchars //        % C++ comments
   \commentchars --        % Lua comments
   {\catcode`\%=12 \_ea}\commentchars %%                  % TeX comments
   {\catcode`\#=12 \catcode`\ =13 \_ea}\commentchars#{ }  % bash comments
   \endtt

   There is one limitation when \TeX/ interprets the comments declared by
   \^`\commentchars`. Each block of comments is accumulated to one line
   and then it is re-interpreted by \TeX. So, the ends of lines in the
   comments block are lost. You cannot use macros which
   need to scan end of lines, for example `\begtt...\endtt` inside the comments.
   The character `%` is ignored in comments but you can use `\%`
   for printing or `%` alone for de-activating `\_endpar` from empty
   comment lines.

   Implementation: The \`\commentchars``<first><second>` redefines the \`\_testcommentchars`
   used in \^`\_printverb` in order to it removes the following `\_iftrue`
   and returns `\_iftrue` or `\_iffalse` depending on the fact that
   the comment characters are or aren't present at the beginning of tested line.
   If it is true (`\ifnum` expands to `\ifnum 10>0`) then the rest of the
   line is added to the \`\_vcomments` macro.

   The \`\_hicomments` is `\relax` by default but it is redefined by \^`\commentchars`
   in order to keep no-colorized comments if we need to use feature from
   \^`\commentchars`.

   The accumulated comments are printed whenever the non-comment line
   occurs. This is done by \`\_printcomments` macro.
   You can re-define it, but the main idea must be kept: it is printed in the
   group, `\_reloding \_rm` initializes normal font, `\catcodetable0`
   returns to normal catcode table used before `\verbinput` is started, and
   the text accumulated in `\_vcomments` must be printed by
   `\_scantextokens` primitive.
   \_cod -----------------------------

\_def\_vcomments{}
\_let\_hicomments=\_relax

\_def\_commentchars#1#2{%
   \_def\_testcommentchars ##1##2##3\_relax ##4\_iftrue{\_ifnum % not closed in this macro
      \_ifx #1##1\_ifx#2##21\_fi\_fi 0>0
      \_ifx\_relax##3\_relax \_addto\_vcomments{\_endgraf}% empty comment=\enfgraf
      \_else \_addto\_vcomments{##3 }\_fi}%
   \_def\_hicomments{\_replfromto{\b\n#1#2}{^^J}{\w{#1#2####1}^^J}}% used in \hisyntax
}
\_def\_testcommentchars #1\_iftrue{\_iffalse} % default value of \_testcommentchar
\_def\_printcomments{\_ttskip
   {\_catcodetable0 \_rm \_everypar={}%
    \_noindent \_ignorespaces \_scantextokens\_ea{\_vcomments}\_par}%
   \_ttskip
}
\_public \commentchars ;

   \_doc -----------------------------
   The \`\visiblesp` sets spaces as visible characters \char9251.
   It redefines the \^`\_dsp`, so it is useful for verbatim modes only.

   The \`\_dsp` is equivalent to {\visiblesp`\ `} primitive.
   It is used in all verbatim environments: spaces are active and defined as
   `\_dsp` here.
   \_cod -----------------------------

\_def \_visiblesp{\_ifx\_initunifonts\_relax \_def\_dsp{\_char9251 }%
                  \_else \_def\_dsp{\_char32 }\_fi}
\_let\_dsp=\  % primitive "direct space"

\_public \visiblesp ;

\_endcode

\_endinput

History:
2022-04-23 ... \_ttline counting: bug fixed
2022-02-22 ... \_reloading removed due to changes in font-select
2021-04-18 ... \_protrudechars=0 added
2021-04-07 ... \_savemathsb, \_restoremathsb introduced
2021-01-22 ... \activettchar changed to \verbchar
2020-12-30 ... \secc followed by \begtt must be unbreakable
2020-11-13 ... \commentchars implemented
2020-04-22 ... \ttshift introduced
2020-04-06 ... \visiblesp added
2020-04-04 ... ^^I activated as \t for multiline verbatim
               \verbinput <cmds> (...) <filename>, <cmds> added.