#StackBounty: #luatex #tex-core #auxiliary-files Problem caching files, aux data not importing

Bounty: 300

The following code caches include. The problem is, for references, counters, toc, to be correct, the corresponding aux file needs to be imported. Unfortunately, I can’t seem to get it to do anything correct(and I have tried many different ways).

Basically when I manually copy and paste the aux file contents to the tex file then compile, I get the desired results, but when I do the equivalent through lua, I get nothing. It is if the inserted tex is completely ignored.

The project is somewhat complex but I will break it down because some people seem to be easily confused or jump to simple conclusion:

  1. This requires windows. Slight modifications should allow it to work on linux or other. The main issue is the hoops required to jump through to do simple file manipulations such as copying due to how poorly TeX is designed for such things.

  2. All the following files go in same directly

TeXCache.lua – Main Lua caching code

if __INCLUDES__TeXCache ~= nil then 
else
    __INCLUDES__TeXCache = true 
    if __DEBUG == true then 
        print("Including TeXCache.lua") 
    end

    require "md5"
    require "lfs"
    require "persist"


    ------------------------------------------ Utility functions
    if not file then file = {} end 
    function file.Exists(name) local f = io.open(name, "r") if f ~= nil then io.close(f) return true end return false end
    function FileExists(name) local f = io.open(name, "r") if f ~= nil then io.close(f) return true end return false end
    function FileSize(name) local f = io.open(name, "r") if f == nil then return -1 end local current = f:seek() local size = f:seek("end") f:seek("set", current) io.close(f) return size end

    function split(str, pat) local t = {} local fpat = "(.-)" .. pat local last_end = 1 local s, e, cap = str:find(fpat, 1) while s do if s ~= 1 or cap ~= "" then table.insert(t,cap) end last_end = e+1 s, e, cap = str:find(fpat, last_end) end if last_end <= #str then cap = str:sub(last_end) table.insert(t, cap) end return t end   
    function string.trim(s, c) if not c then c = " t" end return s:match('^['..c..']*(.-)['..c..']*$') end
    function string.split(str, sep, n) 
        local sep, fields = sep or ":", {}      
        --local pattern = string.format("([^%s]+)", sep) 
        local pattern = '[^'..sep..']'
        if n == 0 or n == nil then pattern = '('..pattern..'+)' end
        if n > 0 then pattern = '^'..(pattern..'+'):rep(n)..'('..pattern..'+)' end
        str:gsub(pattern, function(c) fields[#fields + 1] = c end) 
        return fields 
    end

    function DeepCopyTable(orig)
        local orig_type = type(orig)
        local copy
        if orig_type == 'table' then
            copy = {}
            for orig_key, orig_value in next, orig, nil do
                copy[DeepCopyTable(orig_key)] = DeepCopyTable(orig_value)
            end
            setmetatable(copy, DeepCopyTable(getmetatable(orig)))
        else -- number, string, boolean, etc
            copy = orig
        end
        return copy
    end


    function CopyFile(source, dest)
        local infile = io.open(source, "rb")
        instr = infile:read("*a")
        infile:close()
        local outfile = io.open(dest, "wb")
        outfile:write(instr)
        outfile:close()
    end

    function ReadFile(source, text)
        if not text then text = "rb" else text = "r" end
        local infile = io.open(source, text)
        if not infile then return nil end
        instr = infile:read("*a")
        infile:close()
        return instr
    end

    function WriteFile(source, data)
        local outfile = io.open(source, "wb")
        instr = infile:write(data)
        outfile:close()
    end

    function DeleteFile(Filename)
        -- windows only. os.remove has problems!?
        os.execute('del /Q "'..Filename:gsub("/","\")..'" > nul')
    end


    --------------------------------------------- Setup



    TeXCache = {}
    TeXCache.Arguments = "";
    TeXCache.ModeAfter = "\batchmode"
    TeXCache.Disabled = false
    TeXCache.FileExt = "_TeXCache.tex"
    TeXCache.CacheDir = "TeXCache/"
    TeXCache.HistoryFilename = "TeXCacheHistory.dat";   
    TeXCache.Header = ""        
    TeXCache.ForceRegenerate = false
    TeXCache.Count = 0  
    TeXCache.History = {}
    TeXCache.Temp = {}

    --------------------- Preprocessing TeX
    function StartProcessingTeX()

        -- Load TeXCache.History and set all values to nil, If they are cached, they will be set to 1 at the end and the nil values keys/files can be removed
        TeXCache.History = persistence.load(TeXCache.Path..TeXCache.CacheDir..TeXCache.HistoryFilename);        
        if TeXCache.History then
            if not TeXCache.History.Cached then
                TeXCache.History.Cached = {}
            end         
        else
            TeXCache.History = {}
            TeXCache.History.Cached = {}
        end

        local curPage = -1
        local line 

        -- Read the synctex file and attempt to match input file with pages, onces the pages that the input file spans is determined,
        TeXCache.Temp.ValidCache = false

        TeXCache.Temp.Synctex = {}
        TeXCache.Temp.Synctex.Input_ShortFiles = {}
        TeXCache.Temp.Synctex.Input_Indexes = {}
        -- Read Synctex If exists, else cannot cache. Stores page relations to input file relations
        if FileExists(TeXCache.Path..TeXCache.JobName..".synctex") then 
            local f = io.open(TeXCache.Path..TeXCache.JobName..".synctex", "r")
            if f ~= nil then    
                while true do
                    line = f:read()
                    if line == nil then break end

                    -- Get Input files and store in three way lookup
                    if (string.sub(line,1,6) == "Input:") then
                        local idx = tonumber(split(line, ":")[2])
                        line = string.sub(line,8 + #tostring(idx),-1)
                        line = string.gsub(string.gsub(line, "\./", "\"), "/", "\")
                        local files = split(line, "\")
                        local file = files[#files]
                        local file2 = split(file, "%.") 
                        -- remove tex extension as it is default(we assume that inputs must have an extension of some type, else collisions may occur)
                        if file2[#file2] == "tex" then                                      
                            file2[#file2] = nil                 
                            if #file2 > 1 then
                                file2 = table.concat(file2, ".")
                            else
                                file2 = file2[#file2]
                            end
                            file = file2
                        end

                        TeXCache.Temp.Synctex.Input_ShortFiles[file] = {}
                        TeXCache.Temp.Synctex.Input_ShortFiles[file].idx = idx                  
                        TeXCache.Temp.Synctex.Input_ShortFiles[file].file = file
                        TeXCache.Temp.Synctex.Input_ShortFiles[file].min = 100000
                        TeXCache.Temp.Synctex.Input_ShortFiles[file].max = -1
                        TeXCache.Temp.Synctex.Input_Indexes[idx] = {}
                        TeXCache.Temp.Synctex.Input_Indexes[idx].idx = idx
                        TeXCache.Temp.Synctex.Input_Indexes[idx].file = file
                        TeXCache.Temp.Synctex.Input_Indexes[idx].min = 100000
                        TeXCache.Temp.Synctex.Input_Indexes[idx].max = -1                       

                    elseif (string.sub(line,1,1) == "{") then
                        curPage = string.sub(line,2,-1);
                    elseif #line > 1 then
                        local b = string.sub(line,1,1)
                        if b == "[" or b == "(" or b == "v" or b == "h" or b == "x" or b == "k" or b == "g" or b == "$" then                        
                            local idx = tonumber(split(string.sub(line,2,-1), ",")[1])
                            if not TeXCache.Temp.Synctex.Input_Indexes[idx] then TeXCache.Temp.Synctex.Input_Indexes[idx] = {} end
                            local lp = math.min(TeXCache.Temp.Synctex.Input_Indexes[idx].min or 100000, curPage)
                            local hp = math.max(TeXCache.Temp.Synctex.Input_Indexes[idx].max or -1, curPage)
                            local file = TeXCache.Temp.Synctex.Input_Indexes[idx].file
                            if file then                                 
                                TeXCache.Temp.Synctex.Input_Indexes[idx].min = lp
                                TeXCache.Temp.Synctex.Input_Indexes[idx].max = hp

                                TeXCache.Temp.Synctex.Input_ShortFiles[file].min = lp                                                   
                                TeXCache.Temp.Synctex.Input_ShortFiles[file].max = hp
                            end                         
                        end                                 
                    end
                end                     
                TeXCache.Temp.Synctex.Exists = true
            end     
            f:close()   
        end     


    end 

    function PreBeginDocument()
    end

    function TocWriteAux()
        -- Import aux file preambles headers
        for key, value in pairs(TeXCache.History.Cached) do                                         
            local s = ReadFile(TeXCache.Path..TeXCache.CacheDir.."Cached_"..key..'.aux', 1)
            if s then                   
                local loc = string.find(s, "\@setckpt{")
                s = s:sub(1,loc-1)
                tex.print("\makeatletter")
                tex.print(s)
                tex.print("\makeatother")
                print("nnnrn\makeatletterrn"..s.."rn\makeatotherrnnnn")
            end
        end     
    end

    --------------------- End Of Processing -------------------------
    function DoneProcessingTeX()


        persistence.store(TeXCache.Path..TeXCache.CacheDir..TeXCache.HistoryFilename, TeXCache.History);
        --print("nTABLE----------------------------"); print(inspect(TeXCache.History)); print("nTABLE----------------------------------");
        print("n----Total Includes Cached = "..TeXCache.Count.."n")

        if TeXCache.Backups then
            for key, value in pairs(TeXCache.Backups) do
                -- Create backup of pdf for later use, could optimize duplicate files away
                -- We must redirect the copy to a batch file asychronously because TeX people suck ass(too lame)
                --CopyFile(TeXCache.Path..TeXCache.JobName..".pdf", TeXCache.Path..TeXCache.CacheDir.."Cached_"..key..".pdf")
                if value == 1 then
                    os.execute('Start "copy" /min '..TeXCache.Path..'CopyFile.bat "'..TeXCache.Path..TeXCache.JobName..".pdf"..'" "'..TeXCache.Path..TeXCache.CacheDir.."Cached_"..key..'.pdf"')
                    os.execute('Start "copy" /min '..TeXCache.Path..'CopyFile.bat "'..TeXCache.Path..key..".aux"..'" "'..TeXCache.Path..TeXCache.CacheDir.."Cached_"..key..'.aux"')
                end             
            end
        end

        -- Another hoop to jump through because of the ignorant TeX people. syntex(busy) won't be renamed when using os.execute above for some ignorant reason      
        os.execute('Start "copy" /min '..string.gsub('CopyFile "'..TeXCache.Path..TeXCache.JobName..'.synctex(busy)" "'..TeXCache.Path..TeXCache.JobName..'.synctex"', "/", "\"))      
    end


    -- Function callbacks must be above 
    luatexbase.add_to_callback("stop_run", DoneProcessingTeX, "pdfextract3")    

    -------------------------------------- Caching code






    function TeXCache.Initialize(JobName, path)
        -- To write aux files without getting strange write error one must set MIKTEX_ALLOWUNSAFEOUTPUTFILES=1 in environment for miktex.
        -- Get base_dir using dos if necessary
        if not path then local p = io.popen("echo %cd%") if p then path = p:read("*l"):trim("\").."\" p:close() end end
        if path == nil or path == "" then path = lfs.currentdir() end
        path = (path or ""):trim('"' \/').."/"
        JobName = JobName:trim('"' /') or "temp"
        TeXCache.JobName = JobName
        TeXCache.Path = path
        print('Path = '..TeXCache.Path)

        StartProcessingTeX()
    end tex.print("\directlua{TeXCache.Initialize('\jobname', '\BasePath')}")





    function TeXCache.Include(code)

        -- compare hash of included file with previously stored hash, if the same, then use cache
        -- could use filesize/date comparision for quicker tests
        local codefilename = TeXCache.Path..code..".tex"
        local fs = -1
        local hash = -1
        local curIdx = -1
        local lp = -1
        local hp = -1   

        if FileExists(codefilename) then
            fs = FileSize(codefilename)
            if  TeXCache.History.Cached[code] and TeXCache.History.Cached[code].FileSize == fs then
                hash = md5.sumhexa(ReadFile(codefilename))
                if TeXCache.History.Cached[code].Hash == hash then                   
                    local idx = -1
                    if TeXCache.Temp.Synctex.Input_ShortFiles[code] then
                        curIdx = TeXCache.Temp.Synctex.Input_ShortFiles[code].idx
                        lp = math.max(TeXCache.Temp.Synctex.Input_ShortFiles[code].min, TeXCache.History.Cached[code].LowPage)
                        hp = math.max(TeXCache.Temp.Synctex.Input_ShortFiles[code].max, TeXCache.History.Cached[code].HighPage)
                    else
                        lp = TeXCache.History.Cached[code].LowPage
                        hp = TeXCache.History.Cached[code].HighPage
                    end
                    if lp < 0 or hp < 0 or lp >= 99999 then
                        TeXCache.Temp.ValidCache = false
                    else        
                        TeXCache.Temp.ValidCache = true
                    end     
                else
                    TeXCache.Temp.ValidCache = false
                end
            else
                TeXCache.Temp.ValidCache = false
            end
        else
            TeXCache.Temp.ValidCache = false
        end     


        if TeXCache.Temp.ValidCache == false then
            tex.print("\include{"..code.."}")
            if TeXCache.Backups == nil then
                TeXCache.Backups = {}
            end
            TeXCache.Backups[code] = 1
            -- Store Cached information         
            if FileExists(codefilename) then 
                fs = FileSize(codefilename) 
                hash = md5.sumhexa(ReadFile(codefilename))
            end
            if TeXCache.History.Cached[code] then 
                fs = fs
                hash = hash
                curIdx = curIdx
                lp = math.max(lp, TeXCache.History.Cached[code].LowPage or -1);
                hp = math.max(hp, TeXCache.History.Cached[code].HighPage or -1);
            end
            TeXCache.History.Cached[code] = {}                  
            TeXCache.History.Cached[code].FileSize = fs
            TeXCache.History.Cached[code].Hash = hash
            TeXCache.History.Cached[code].LowPage = lp
            TeXCache.History.Cached[code].HighPage = hp     
            return
        end

        for key, value in pairs(TeXCache.History.Cached) do         
            if code == key then
                print("nUsing cached `"..code..".tex!n")
                local filename = TeXCache.Path..TeXCache.CacheDir.."Cached_"..code          
                tex.print("\includepdf[pages={"..lp.."-"..hp.."}]{"..filename.."}")

                -- Import aux file counters                             
                local s = ReadFile(TeXCache.Path..TeXCache.CacheDir.."Cached_"..key..'.aux')
                if s then               
                    local _, loc = string.find(s, "\@setckpt{"..code.."}{")
                    s = s:sub(loc+1, -1)                
                    for i=#s, 1, -1 do
                        if s:sub(i,i) == "}" then
                            s = string.sub(s,1,i-1) 
                            break;
                        end
                    end

                    s = "rn\makeatletterrn"..s.."rn\makeatotherrn"
                    tex.print(s)
                    print("rnrnrn"..s.."rnrnrn")
                end

                -- Store cached info for use on next run
                TeXCache.History.Cached[code].FileSize = fs
                TeXCache.History.Cached[code].Hash = hash
                TeXCache.History.Cached[code].LowPage = lp
                TeXCache.History.Cached[code].HighPage = hp                                     
                TeXCache.Count = TeXCache.Count + 1
                break
            end     
        end



    end


end -- __INCLUDE__

CopyFile.bat – Used to copy files asynchronously

@echo off
@REM use asynchronously to copy a file after a process as exited(set timeout to be long enough for process to end).
@REM Call with Start "copy file" <path to this batch file> <source> <dest>

TIMEOUT 3 /NOBREAK
copy /B /Y %1 %2
exit

Clear.bat – Clears junk files to remove the caching(to start reset everything)

@echo off
del test.synctex(busy)
del test.synctex
del test.pdf
del test.toc
del test.log
del test.aux
del TeXCacheTeXCacheHistory.dat
del TeXCacheCached_testinput.pdf
del TeXCacheCached_testinput.aux

persist.lua – Used to store and retrive lua tables(can get from git hub)

--t_original = {1, 2, ["a"] = "string", b = "test", {"subtable", [4] = 2}};
--persistence.store("storage.lua", t_original);
--t_restored = persistence.load("storage.lua");

local write, writeIndent, writers, refCount;

persistence =
{
    store = function (path, ...)
        local file, e = io.open(path, "w");
        if not file then
            return error(e);
        end
        local n = select("#", ...);
        -- Count references
        local objRefCount = {}; -- Stores reference that will be exported
        for i = 1, n do
            refCount(objRefCount, (select(i,...)));
        end;
        -- Export Objects with more than one ref and assign name
        -- First, create empty tables for each
        local objRefNames = {};
        local objRefIdx = 0;
        file:write("-- Persistent Datan");
        file:write("local multiRefObjects = {n");
        for obj, count in pairs(objRefCount) do
            if count > 1 then
                objRefIdx = objRefIdx + 1;
                objRefNames[obj] = objRefIdx;
                file:write("{};"); -- table objRefIdx
            end;
        end;
        file:write("n} -- multiRefObjectsn");
        -- Then fill them (this requires all empty multiRefObjects to exist)
        for obj, idx in pairs(objRefNames) do
            for k, v in pairs(obj) do
                file:write("multiRefObjects["..idx.."][");
                write(file, k, 0, objRefNames);
                file:write("] = ");
                write(file, v, 0, objRefNames);
                file:write(";n");
            end;
        end;
        -- Create the remaining objects
        for i = 1, n do
            file:write("local ".."obj"..i.." = ");
            write(file, (select(i,...)), 0, objRefNames);
            file:write("n");
        end
        -- Return them
        if n > 0 then
            file:write("return obj1");
            for i = 2, n do
                file:write(" ,obj"..i);
            end;
            file:write("n");
        else
            file:write("returnn");
        end;
        if type(path) == "string" then
            file:close();
        end;
    end;

    load = function (path)
        local f, e;
        if type(path) == "string" then
            f, e = loadfile(path);
        else
            f, e = path:read('*a')
        end
        if f then
            return f();
        else
            return nil, e;
        end;
    end;
}

-- Private methods

-- write thing (dispatcher)
write = function (file, item, level, objRefNames)
    writers[type(item)](file, item, level, objRefNames);
end;

-- write indent
writeIndent = function (file, level)
    for i = 1, level do
        file:write("t");
    end;
end;

-- recursively count references
refCount = function (objRefCount, item)
    -- only count reference types (tables)
    if type(item) == "table" then
        -- Increase ref count
        if objRefCount[item] then
            objRefCount[item] = objRefCount[item] + 1;
        else
            objRefCount[item] = 1;
            -- If first encounter, traverse
            for k, v in pairs(item) do
                refCount(objRefCount, k);
                refCount(objRefCount, v);
            end;
        end;
    end;
end;

-- Format items for the purpose of restoring
writers = {
    ["nil"] = function (file, item)
            file:write("nil");
        end;
    ["number"] = function (file, item)
            file:write(tostring(item));
        end;
    ["string"] = function (file, item)
            file:write(string.format("%q", item));
        end;
    ["boolean"] = function (file, item)
            if item then
                file:write("true");
            else
                file:write("false");
            end
        end;
    ["table"] = function (file, item, level, objRefNames)
            local refIdx = objRefNames[item];
            if refIdx then
                -- Table with multiple references
                file:write("multiRefObjects["..refIdx.."]");
            else
                -- Single use table
                file:write("{n");
                for k, v in pairs(item) do
                    writeIndent(file, level+1);
                    file:write("[");
                    write(file, k, level+1, objRefNames);
                    file:write("] = ");
                    write(file, v, level+1, objRefNames);
                    file:write(";n");
                end
                writeIndent(file, level);
                file:write("}");
            end;
        end;
    ["function"] = function (file, item)
            -- Does only work for "normal" functions, not those
            -- with upvalues or c functions
            local dInfo = debug.getinfo(item, "uS");
            if dInfo.nups > 0 then
                file:write("nil --[[functions with upvalue not supported]]");
            elseif dInfo.what ~= "Lua" then
                file:write("nil --[[non-lua function not supported]]");
            else
                local r, s = pcall(string.dump,item);
                if r then
                    file:write(string.format("loadstring(%q)", s));
                else
                    file:write("nil --[[function could not be dumped]]");
                end
            end
        end;
    ["thread"] = function (file, item)
            file:write("nil --[[thread]]n");
        end;
    ["userdata"] = function (file, item)
            file:write("nil --[[userdata]]n");
        end;
}

test.tex – The demo tex file to test

documentclass[12pt,oneside]{book}
%batchmode

directlua{
    require("debugger")()
}

directlua{
    % Must be ran first, sets up paths
    __DEBUG = true
    local path = BasePath
    local p = io.popen('echo '..string.char(37)..'cd'..string.char(37)) if p and (path == nil or path == "") then path = p:read("*l") p:close() end
    if path == nil or path == "" then path = lfs.currentdir() end
    if path == nil or path == "" then else BasePath = path end
    path = string.gsub(path, string.char(92), "/");
    BasePath = string.gsub(BasePath, string.char(92), "/")
    tex.print(-1, string.char(92).."def"..string.char(92).."BasePath"..string.char(123)..BasePath..string.char(125))        
}

directlua{
    dofile('TeXCache.lua')
}

usepackage{luatex}
usepackage{xcolor}
usepackage{everypage}
usepackage{afterpage}
usepackage{pdfpages}
usepackage{everyhook}
usepackage{ifthen}
usepackage{letltxmacro}
usepackage{atbegshi}
usepackage{lipsum}

directlua{tex.enableprimitives('',tex.extraprimitives())}




newcommand{tableofcontentsEx}{tableofcontentsdirectlua{TocWriteAux()}}
newcommand{cachedInput}[1]{clearpagedirectlua{TeXCache.Include("luaescapestring{#1}"); }clearpage}

title{Sections and Chapters}
author{Test Author}
date{ }




directlua{if PreBeginDocument then PreBeginDocument() end}
begin{document}





maketitle
tableofcontentsEx
clearpage


lipsum

%This either includes or includespdf when cached.
cachedInput{testinput}





slfkljasfasfjllasfjas;df

234214214892784
chapter{HELP!!!}
asdfjhajsdfkljasjdf
a
asdffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff


end{document}

testinput.tex – the demo input text file – can be any normal tex file

chapter{AMAZING!}

{{{}}}{}{}{}{}{}{}
$$x =
y + z + z^2 + z^{311}
$$

begin{tabular}{|c|c|}
  hline
  % after \: hline or cline{col1-col2} cline{col3-col4} ...
  1 & 1 \
  2 & 2 \
  hline



end{tabular}


$$x =
y
$$

$z =
x
$

Alipsum



{

textcolor{red}{
This is a test cache1
Blipsum
textcolor{green}{Clipsum}
}



$$x =
y + z + z^2 + z^43434
$$
lipsum
lipsum
lipsum























$z =
x
$




Dlipsum
test
xxxxxxxxxxxxxxxx

These are all the files one needs. One can find the full project at http://www.filedropper.com/test_192

It would be nice if the code specified line numbers so I could reference the blocks that deal with the aux file. In any case, There are only really two parts in the TeXCache.lua that deal with the aux.

One is to try to load it in the preamble using TocWriteAux() and the other is in the TeXCache.Include function. This occurs where it says “– Import aux file counters”.

What I tried to do was split the aux file up in to what occurs before setckpt and what occurs after because simply using ‘input’ did not work.

One can remove the aux code and empty TocWriteAux() to not have any aux behavior from the lua code. I only included it so one could potentially modify it if there is an issue.

Again, the main issue is that when I “tex.print” the aux code in lua it doesn’t work yet it works when I copy and paste. Why? I have no idea! That is why I am asking. Please don’t “help” with generic answers but by providing a solution that makes the test project work as expected(any answers that do not solve this problem are not answers. It should also work in general(at least 95% of test cases.)). I’ve went out of my way to provide a through “MWE” and I expect anyone trying to help to do the same rather than trying to get a “gold star” or whatever.

So, what is the expected behavior?

  1. The toc is always correct. I realize that the way the toc is that it might require an extra compilation for it to update properly but that is a general issue with the toc design and not the caching mechanism.
  2. The chapter counters are always right. They are right after the first run but when the caching mechanism kicks in, they are wrong because the aux file is not being used. Other counters and such should also be properly imported. The whole point of the aux file is to keep such things in sync, and so it should behave as it should regardless of the caching design.

e.g., after compiling test a few times, The “Help” chapter is 1 when it should be 2. One can insert the aux file directly(copy and paste_ and things work. So why the lua code, which essentially does the same thing(at least what I tried first before I tried to split), does not work is beyond me.

  1. The lua code should not be necessary to import the aux info? I have been told this but the methods I tried did not work. e.g., after the line

    tex.print(“includepdf[pages={“..lp..”-“..hp..”}]{“..filename..”}”)

one can add

tex.print("\input{"..filename..".aux}")

and it should import the aux file, but it does nothing. Just as if one added it directly to the tex.

Anyways, hopefully someone will put in the time to get this to work. Being able to cache includes is a nice feature and quite a speed saver for slow projects. While it is a bit too granular compared to per page caching, it is better than nothing and avoids the “includeonly” bs that we currently use.

I’d expect the answer to be simple as it is just doing something that tex already does… e.g., include the aux file. But either I am screwing that up or it simply is not working as it should(a bug or due to some weird issue with the TeXCache code).

Note, another bug is that the synctex file is not closed properly. Why that occurs seems to be another bug. It occurs only because I delegate the copying of the pdf file asynchronously to the os. This is because there is no post pdf callback. If one tries to copy the pdf at the “end” through lua(either the stop_run callback or whatever) the pdf is not completely finished and is invalid. These issues and workarounds are due to the cobbled nature of TeX, LuaLatex, and syntex. If a proper callback hierarchy was added along with a few other things some very good features could be created. (e.g., nearly a transparent full page by page caching mechanisms, etc)

Anyways, thanks for any help…


Get this bounty!!!

Leave a Reply