Findent in depth

Following is the doc/README file:

For installation instructions and usage, see README.1st and INSTALL

This document is a not very well structured story about findent.
You can skip this and use findent after reading the output of
  findent -h.

findent, what?
==============

   findent is an indenter for Fortran programs, fixed and free format.
   findent can also translate fixed format to free format and vice versa.
   Since version 3.0.0, findent can generate dependencies based on USE,
   MODULE, SUBMODULE, INCLUDE, #include, ??include  and emit a bash script
   that, using findent, creates a dependency file to be used in a Makefile.

   findent will take care of:

      continuation lines
      multi-statement lines
      labelled and unlabelled do-loops
      IF
      IF ... THEN ... ENDIF
      where
      FORALL
      WHERE constructs
      FORALL constructs
      etc. see findentclass.cpp for details

   findent will remove trailing spaces and tabs, and convert
   tabs at the start of a line into spaces. By default, statement 
   labels are placed at the start of a line. Apart from this and
   indenting, findent will not alter the input, trying to
   preserve alignment. For example, the alignment in:

      X = 3.0*A + 4*B +  &
      &   2  *C +   Y

   will remain intact.

   Optionally, findent will refactor lines that end a subroutine etc.:

      SUBROUTINE SUB
      ...
      END FUNCTION MYFUN

   will become:

      SUBROUTINE SUB
      ...
      END SUBROUTINE SUB

   findent is space-insensitive, for example a line like:

      REALFUN  CTIONFUN(X)

   is recognized as the start of a function definition.

   Usage:

       findent -h

findent, why?
=============

   There are a number of public domain Fortran indenting tools, for example:

   - vim is shipped with an simple Fortran indenter
   - emacs has a Fortran indenter
   - floppy, only for fixed format: http://www.netlib.org/floppy/
   - convert.f90: converts from fixed to free format, and indents:
        ftp://ftp.numerical.rl.ac.uk/pub/MandR/convert.f90
   - f2f90: based on convert.f90: http://www.fortran.com/f2f90.tar.gz
   - f90ppr: an impressive piece of software that beautifies
        Fortran code and contains a macro processor.
        http://fortranwiki.org/fortran/show/f90ppr
   - fprettify: an indenter and beautifier, written in Python.
    IMHO not mature at the time of this writing (august 2018).
        https://github.com/pseewald/fprettify

   For me, the problem with these tools is, that 

    - they are too simple (for example, do not recognize labelled 
      do-loops) 
    - or do too much (destroying neatly aligned pieces of code)
    - or are for me too complicated to adapt and extend.

   Furthermore, I want that indenting does not make irreversible changes 
   to the source: I want always be able to get back to the version after
   the first indenting. (Exceptions: converting from fixed to free format;
   adding 'subroutine foo' after 'end').

   Therefore I decided, having some spare time after my retirement, 
   to try to build a Fortran indenter, based on flex and bison for
   readability.
   As programming language I chose C++, because of the availability
   of string and deque.


findent, how?
=============

   So, here it is, a Fortran indenter to my taste, based on flex,
   bison and g++. 

   The program performs the following major tasks:

    - determine the input format: free or fixed
    - glue together continuation lines removing comments
    - pre-process the assembled input line, to make it better processable 
        by flex: remove white space, substitute strings, hollerith's, 
    statement label and operators like .EQ. by special tokens
    - perform a two-stage parsing:
      - try if the line is an assignment
      - if it is not an assignment, parse the line using as tokens the
        Fortran keywords (SUBROUTINE, DO, ...)
    - based on the outcome of the parse, determine the indentation
    - output the lines that were read in to compose the full line,
      trying to preserve the lay-out after the original leading white
      space, optionally converting from fixed-form to free-form.
      Also optionally, lines that end a subroutine, program etc.,
      are completed (or even modified) as in:
        END subroutine mysub
      Preprocessor statements are accounted for to prevent that code like:

        #ifdef one
        SUBROUTINE ONE
        #else
        SUBROUTINE TWO
        #endif

       would result in a double SUBROUTINE indentation.
       Moreover, track is kept of do-labels, in order to correctly indent
       constructs like:

           DO 10 I=1,20
         DO 10 J=1,10
        X(I) = Y(I)+J
        10 CONTINUE

Findent, structure
==================

   In version 2.8.4, a major reorganizing has been done, trying to
   make the source more readable and more object-oriented.

   These are the classes (all starting with a capital):

   - Debugostream

      This class contains code for debugging.

   - Docs

      This class contains code (often generated) to output help-
      texts (includeing this text) etcetera.

   - Findentclass

      Findentclass contains basic functions and variables that
      are used throughout the program, such as:

      int determine_fix_or_free()
         if the input format is not given (parameter -ifixed
     or -ifree), this function tries to find a proof that
     the program is free-format. If so, FREE is returned,
     else FIXED.

      Fortranline mygetline()
         Reads a line from STDIN.
     Optionally, the function builds a buffer to store the
     line just read, this option is used by determine_fixed_or_free().

      Fortranline Getline()
         Uses the buffer from mygetline() or mygetline() to return
     a Fortranline. Optionally, maintains a buffer (wizardbuffer)
     to enable a look-ahead for the wizard functions.


   - Fortranline

      This class contains a line of fortran code, and has many
      frequently used functions operating on that line.

   - Fortran

      This is an abstract class with functions and variables needed
      to construct a 'full_statement': a string that is composed
      of pure fortran code (stripped from comments, preprocessor
      directives, ..) and a deque of lines that constructed the
      full_statement, including comments and preprocessor statements. 

      The pure virtual functions are filled in by class Fixed or
      class Free, depending on the format of the input.

      The indentation required is computed from full_statement
      and the current indentation.

      Special provisions are made for storing labelled do statements,
      preprocessor statements and more.

   - Free

      This class is a child of Fortran, and contains functions to
      indent and output the deque containing the continuation lines
      of a free format fortran source.
      Also, when conversion from free to fixed format is required, 
      there is code to store the continuation lines in a fixed
      (not properly indented) deque, and present this to Fixed to
      output this in a properly indented format.

   - Fixed

      Basically the same as Free, but vice-versa. 
      There is a special issue: to determine if there are continuation
      lines, a wizard has been created that can look ahead, see
      getnext() in Findentclass.

   - Flags

      This class implements a container for the flags (-ifree, -i4 ...)
      and contains code to interpret the flags.

   - Globals

      This class implements a container for a few global variables.
      Maybe, one would be tempted to make these static, but I tried 
      not to use static variables in order to able to run more than
      one Findent in one program, one reading from STDIN, the other
      from a file for example. 

   - Line_prep

      This class contains code to pre-analyze a full_statement to
      make the work easier for the lexer and parser.

   - Pre_analyzer

      This class contains code to see if a line is a preprocessor
      statement or a findentfix: line.

   - Simpleostream

      This class implements a simple filter to std::cout.

   The program starts in findent.cpp, and goes on in fortranrun.cpp.


   The whole thing is somewhat more hairy than I would like to see.
   I did not succeed to make the whole thing strictly hierarchical,
   and solved this by giving classes access to the internals of
   other classes using pointers. But, like in biology, in creating
   a program, there is no law to keep it simple.


findent, usage?
===============

   Findent reads from standard input and writes to standard output:

      findent < prog.f90 > prog1.f90

   See also 'wfindent', later in this file.

   The command

      findent -h

   gives an overview of the possible flags and there effect. Most
   interesting are:

      -i<n>  
        example: -i5
        which determines the amount of indent to be used by every
        item that calls for indenting
      -Ia
        The starting indent is determined from the first line (more
        or less), useful when using findent within vim, for example
        to intent a selected region:

           '<,'>:!findent -Ia

      -ofree
        converts from fixed format to free format.

      -L<n>
        example: -L72
        limit input line length to 72 characters.

   NOTE 1: findent knows about tabbed input: for fixed-format input,
           the following transformations are made:

           10<tab>I=   -> 10<sp><sp><sp><sp>I=
           <tab>1K*J   -> <sp><sp><sp><sp><sp>1K*J
           <tab>X=Y    -> <sp><sp><sp><sp><sp><sp>X=Y

           So, a tab followed by 1-9 is transformed to a continuation line,
           otherwise to a normal line, starting in column 7.

   NOTE 2: findent silently ignores errors in the flags

   NOTE 3: handling of continuation lines
           Example:

              a = &
                 (/ 3, 10, 12, 4, &
                    5,  9,  1, 0, &
                   13,  2, 25, 6 /)

           After running findent, with standard parameters, you get this:

              a = &
                 (/ 3, 10, 12, 4, &
                 5,  9,  1, 0, &
                 13,  2, 25, 6 /)

          That is probably not what you really want.

          The recommended solution is: add '&' at the start of the 
          continuation lines:

           a = &
              &   (/ 3, 10, 12, 4, &
              &      5,  9,  1, 0, &
              &     13,  2, 25, 6 /) 

          Findent will indent this as:

           a = &
           &   (/ 3, 10, 12, 4, &
           &      5,  9,  1, 0, &
           &     13,  2, 25, 6 /)

         Not recommended solution: You can use the '-k-' flag, like:
           findent -k- < prog.f90 > prog1.f90

         Findent will in this case not touch continuation lines without 
         a starting '&', but leave them as they are.

   NOTE 4:

        Findent does not check the length of an output line, so it could
        be that the length will be larger than 72 or 132 for fixed and 
        free format respectively. In fact, indenting old fixed format
        sources will very likely result in lines longer than 72 columns.
        If you are lucky, the compiler will generate an error message,
        but too long lines can result in changing the semantics of a 
    program without notice.
        Advice: use a compiler flag that allows long lines:

          gfortran, free format:    -ffree-line-length-none  # unlimited
          gfortran, fixed formtat:  -ffixed-line-length-none # unlimited
          ifort, free format:       # no flag needed, default is unlimited
          ifort, fixed format:      -132  # max line length is 132
          pgf90, free format:       # max line length is 264, 
                                # error if longer
          pgf90, fixed format:      -Mextend  # max line length is 132, 
                                    #           no error if longer

   NOTE 4.1:
        Here a script to check for line length:

>>>> snip ---------- checklength ---------------------------------------


#!/bin/bash
# checks file line lengths
# Usage:
# checklength <length> [file ...]
# outputs "filename:line number:line length:line" for lines longer than length
# tabs are converted to spaces using expand
# if no file is given, read from stdin
usage()
{
   echo "Usage:"
   echo "$0 <length> [file ...]"
}
if [ -z "$1" ] ; then
   usage
   exit 1
fi
l="$1"
doit()
{
   expand | awk -v l="$1" -v f="$2" '{
   if (length($0) > l)
     printf "%s:%d:%d:%s\n",f,FNR,length($0),$0
   }'
}

if [ -z "$2" ] ; then
   doit "$l" "-"
   exit 0
fi

shift
while [ "$1" ] ; do
   cat "$1" | doit "$l" "$1"
   shift
done

<<<< snip ---------- checklength --------------------------------------

   Example of usage:
       checklength 72 *.f

   NOTE 5: handling of comment lines

        Findent indents comment lines, but does not touch comment lines
        with the '!' in column one.

   NOTE 5.1: handling of comment lines converting fixed to free format

        As said above, findent does not touch comments starting
        in column 1. Since all vintage comments start in column 1,
        this has the effect that these comments will not be indented
        when converting from fixed to free format (using -ofree).
        If you want the comments indented, convert to free format,
        add a space before every line and use findent again.

        Example if the stream editor 'sed' is available:

          findent -ofree < prog.f | sed 's/^/ /' | findent > prog.f90

        If 'sed' is not available (on Windows for example), you
        can create the program 'addspace' or, on Windows, 'addspace.exe'
        by compiling this program:

>>>> snip -------- addspace.f ------------------------------------------
program addspace
   implicit none
   character(1000) :: line
   integer         :: io
   do
      read(*,'(a)',iostat=io) line
      if (io .ne. 0) exit
      write(*,'(1x,a)') trim(line)
   enddo
end program addspace
<<<< snip -------- addspace.f ------------------------------------------

        Or, if you have a vintage Fortran-4 compiler, by compiling
        this program:

>>>> snip -------- ADDSPACE.F ------------------------------------------
      DIMENSION L(1000)
      DATA LB/1H /
   10 DO 15 I=1,1000
   15 L(I)=LB
      READ(5,100,END=30) L
      DO 20 I=1000,1,-1
         IF (L(I)-LB) 25,20,25
   20 CONTINUE
      WRITE(6,110)
      GOTO 10
   25 WRITE(6,110) (L(J),J=1,I)
      GOTO 10
   30 CONTINUE
  100 FORMAT(1000A1)
  110 FORMAT(1H ,1000A1)
      END
<<<< snip -------- ADDSPACE.F ------------------------------------------

        Use the generated program 'addspace' in stead of 'sed':

        findent -ofree < prog.f | addspace | findent > prog.f90

findent, failure, findentfix:
=============================

   One thing is certain: findent contains errors. I appreciate it 
   if you bring errors to my attention. If possible I will fix them.

   On the other hand, it is possible to fool findent, for example
   by using #ifdef, #else, #endif in a way that confuses findent.

   Both cases can be solved using ! findentfix: , read on:

   The next program will not be indented correctly:

>>>> snip -------- fixdemo.f90 -----------------------------------------
! compile with: gfortran -cpp fixdemo.f90
! or
!               gfortran -cpp -DLOOPJ fixdemo.f90
   program fixdemo
      implicit none
      integer i,j
      j=4
      do i=1,3
#ifdef LOOPJ
      do j=1,2
#endif
      print *,i*j
      enddo
#ifdef LOOPJ
      enddo
      print *,'with j-loop'
#else
      print *,'without j-loop'
#endif
      continue
   end program fixdemo
<<<< snip -------- fixdemo.f90 -----------------------------------------

   That is because findent takes the indentation from:
#ifdef LOOPJ
      do j=1,2
#endif
   and
#else
      print *,'without j-loop'
#endif

   So, findent is missing an enddo for the j-loop.

   In this case, the solution would be to insert #else just before the
   first #endif. 
   If, however, in a real-world example this is not possible, or 
   when findent really makes an error, you can use findentfix.
   In the example above, insert directly after the last #endif:
     ! findentfix: enddo
   and findent will indent correctly. In general, the text after
     ! findentfix:
   will be used by findent as a normal source line, so the following
   could also be useful:
     ! FINDENTfix: subroutine dummy
     ! findentFIX: do;do;do
     ! FINDENTFIX: end;end
     ! findentfix: where ()
   But the following would do nothing:
     ! findentfix: continue


findent, creating a dependency file for use in an Makefile.
===========================================================

      findent --deps < prog.f90

   prints the dependencies found in prog.f90, based on USE,
   MODULE, SUBMODULE, INCLUDE, #include and ??include.

   This is used by the bash script makefdeps to create a dependency
   file for use in an Makefile. 
   Creation of the makefdeps script:

      findent --makefdeps > makefdeps
      chmod +x makefdeps

   The command:

      ./makefdeps *.f90

   will output a dependency file, to be used in an Makefile.
   The dependencies are based on .o files, for example:

      main.o: sub1.o sub2.o
      sub1.o: sub1.inc

   Example Makefile and fortran sources to create 'program' from 
   main.f90 sub1.f90 sub2.f90 sub.inc:


>>>> snip ---------- Makefile ---------------------------------------
SRCS = main.f90 sub1.f90 sub2.f90
OBJS = $(SRCS:.f90=.o)
%.o: %.f90
    gfortran -c -o $@ $<
program: $(OBJS)
    gfortran -o $@ $(OBJS)
include deps
dep deps:
    findent --makefdeps < /dev/null > makefdeps.sh
    @if [ "`head -n 1 makefdeps.sh`" != "#!/bin/bash" ] ; then \
       findent -v 1>&2; \
       echo "Use findent version >= 3.0.0" 1>&2 ; exit 1 ; fi
    chmod +x makefdeps.sh
    ./makefdeps.sh $(SRCS) > deps
clean: 
    rm -f *.o *.mod *.smod program deps
<<<< snip ---------- Makefile ---------------------------------------

>>>> snip ---------- main.f90 ---------------------------------------
   program main
      use mymod1
      use mymod
      call sub1
      call sub
   end
<<<< snip ---------- main.f90 ---------------------------------------

>>>> snip ---------- sub1.f90 ---------------------------------------
   module mymod1
   contains
      subroutine sub1
         print *,'this is sub1'
      end
   end
   module mymod
      interface
         module subroutine sub
         end subroutine
      end interface
   end module
<<<< snip ---------- sub1.f90 ---------------------------------------

>>>> snip ---------- sub2.f90 ---------------------------------------
   submodule (mymod) mymod2
   contains
      module procedure sub
   include 'sub.inc'
      end procedure
   end submodule
<<<< snip ---------- sub2.f90 ---------------------------------------

>>>> snip ---------- sub.inc ---------------------------------------
print *,'this is sub'
<<<< snip ---------- sub.inc ---------------------------------------

   The fortran sources will be compiled in correct order, you can even
   use parallel make (make -j). BTW: this example uses the SUBMODULE
   statement: you need to have gfortran >= 6 to compile.


findent, installation:
======================

   (For a more comprehensive text, see README.1st and INSTALL)

   Linux:
         $ ./configure --prefix=/usr/local
         $ make
     On systems with sudo (Debian, Ubuntu, Mint, ...):
     $ sudo make install
     On systems without sudo (Redhat, Scientific Linux, ...):
     $ su -c 'make install'

   Windows:
      copy findent.exe C:\WINDOWS

wfindent
========

  wfindent, a bash shell script, indents Fortran source in-place, 
  performing a sanity check.
  It accepts all flags that findent accepts.

  Usage:

     wfindent [ findent flags ] files

   example

     wfindent -I4 *.f90

   Installation:

      If you installed findent with the ./configure, make, make install 
      method, wfindent is installed as well.
      Otherwise:
      On systems with sudo (Debian, Ubuntu, Mint, ...):
      $ sudo install scripts/wfindent /usr/local/bin
      On systems without sudo (Redhat, Scientific Linux, ...):
      $ su -c 'make install'

wfindent.bat
============

  wfindent.bat is for usage in the cmd shell of Windows and has the same
  functionality as wfindent, described just above.

  Installation:

    copy wfindent.bat C:\WINDOWS


jfindent or jfindent.jar
========================

  jfindent is a graphical front end for findent, and is available as
  a separate package on findent.sourceforge.net

findent and vim
===============

   Findent is since version 2.7 very vim-aware. When using the vim scripts
   (look at findent --vim-help), findent is used as equalprg 
   ( :help equalprg )
   and indentexpr ( :help indentexpr )
   This is possible because of the speed of findent: it indents about 
   50000 - 100000 lines per second.

   Findent can emit configuration files, look at the output of:

      findent --vim_help

findent and gedit
=================

   To enable findent in gedit, look at the output of:

      findent --gedit_help

findent and emacs
=================

   To enable findent in emacs, look at the output of:

      findent --emacs_help

Issues
======

   Since findent parses line-by-line, there are situations that are 
   ambiguous:

     F(X) = X**2        An assignment or a statement function?

     ELSE WHERE          Is this an ELSEWHERE as in
                        WHERE(X .EQ. 0)
                           Y=10
                        ELSE WHERE
                           Y=1
                        END WHERE

                         or is it part of an IF construct with name WHERE:
                        WHERE: IF (X .EQ. 0) THEN
                                  Y=10
                               ELSE WHERE
                                  Y=1
                               ENDIF WHERE

                         Findent chooses the first possibility.

   And there must be more ...
   Luckily, it seems that these ambiguities do not affect indentation.

   I tried to make findent Fortran-2008 compatible. This raised another
   ambiguity, because findent is space-insensitive:

   MODULE PROCEDURE MYPROC  Is this an module PROCEDUREMYPROC or
                            an moduleprocedure MYPROC?
                            Findent assumes the last.

Thanks to
=========

It would not be possible for me to create findent if I could not stand
of the shoulders of other free software projects. In alphabetical order
(and probably forgetting some):

autoconf:      https://www.gnu.org/software/autoconf/autoconf.html
bash:          https://www.gnu.org/software/bash/
bison:         https://www.gnu.org/software/bison/
debian:        https://www.debian.org/
flex:          https://github.com/westes/flex
g++:           https://gcc.gnu.org/
gcc:           https://gcc.gnu.org/
gfortran:      https://gcc.gnu.org/wiki/GFortran
gnu software:  https://www.gnu.org/
kcachegrind:   https://kcachegrind.github.io/html/Home.html
linux:         https://en.wikipedia.org/wiki/Linux
make:          https://www.gnu.org/software/make/
ubuntu:        https://www.ubuntu.com/
valgrind:      http://valgrind.org/
vim:           https://www.vim.org/

The website http://www.cplusplus.com/ helped me to find a way through C++.

I like to thank the people who gave suggestions to improve the functionality
of findent.

=============================================================================

I am happy to receive comments, error reports and suggestions for
improvements.

October 2018, Willem Vermin, wvermin@gmail.com