Author Topic: overcome size limits (USHRT_MAX) in tridscan.py triddefspack.py  (Read 546 times)

jenderek

  • Sr. Member
  • ****
  • Posts: 375
overcome size limits (USHRT_MAX) in tridscan.py triddefspack.py
« on: February 08, 2024, 10:19:42 PM »
Hello trid users,

some days ago i read an article about emulation of old commodore computer.  In
that context disc image with D64 file name extension are mentioned.

So i run trid utility on such examples with D64 suffix. The samples are
"recognized" and described wrong as "null bytes" because many samples contain
nil byte at the beginning. Or samples are described wrong as "MacBinary 1",
"Adobe PhotoShop Brush" "Sybase iAnywhere database files" because the
corresponding definitions contains only few and nil bytes at the beginning.
(see appended trid-v-old.txt in output).

For comparison reason i also run the file format identification utility DROID
(See https://sourceforge.net/projects/droid/). Here the samples are also not
recognized.

For comparison reason i also run file command (version 5.45) on such samples.
Here most of the samples are recognized and most are described as "D64 Image"
(see appended file-k-5.45.txt in output).

Luckily i found about D64 on file formats archive team web server. So this
expressed inside new TrID definition d64.trid.xml by line like:
   <RefURL>http://fileformats.archiveteam.org/wiki/D64</RefURL>

But when i refine definition by running tridscan on more samples than most
patterns vanish and i get few nil byte patterns at the beginning like:
   <Pattern>
      <Bytes>00</Bytes>
      <Pos>0</Pos>
   </Pattern>
   <Pattern>
      <Bytes>00</Bytes>
      <Pos>256</Pos>
   </Pattern>
   <Pattern>
      <Bytes>00</Bytes>
      <Pos>512</Pos>
   </Pattern>
   <Pattern>
      <Bytes>00</Bytes>
      <Pos>768</Pos>
   </Pattern>
   <Pattern>
      <Bytes>00</Bytes>
      <Pos>1024</Pos>
   </Pattern>
   <Pattern>
      <Bytes>00</Bytes>
      <Pos>1280</Pos>
   </Pattern>
   <Pattern>
      <Bytes>00</Bytes>
      <Pos>1536</Pos>
   </Pattern>
   <Pattern>
      <Bytes>00</Bytes>
      <Pos>1792</Pos>
   </Pattern>

But when running here tridscan with samples brucelee.d64 i get error message
like:
TrIDScan/Py v2.02 - (C) 2015-2016 By M.Pontello

File(s) to scan found: 1
Scanning for patterns...
Checking file 1/1 '.\brucelee.d64'
tridscan.py: Error: no patterns found!

But i know that there must exist some shared patterns, because file command
recognize most examples. So i looked inside Magdir/c64 fragment of file
sources and looked how there the recognition happens. There at offset 16500h
(=91392 decimal) check for 4 byte sequence 12014100 is done. So i create
manually a definition like d64-bad.trid.xml with translated pattern. when now
running triddefspack i get error message like:
Building files list...
No update!
Found 12 definitions.
Reading...
Packing...
Traceback (most recent call last):
  File "C:\PROGS\trid\triddefspack.py", line 362, in <module>
    main()
  File "C:\PROGS\trid\triddefspack.py", line 329, in main
    defdatanew = trdbuild(deflist, CMD["strip"])
  File "C:\PROGS\trid\triddefspack.py", line 235, in trdbuild
    defsdata.append(trddef2bin(triddef, stripflag))
  File "C:\PROGS\trid\triddefspack.py", line 207, in trddef2bin
    PatBlock.append(pack("<HH", pos, len(patbytes)) + patbytes)
struct.error: ushort format requires 0 <= number <= USHRT_MAX

So is it possible to change some size limits inside TrID tools that D64 disc
images can be recognized?

I would be pleased to get at least some hints. Thanks
Jörg Jenderek

Mark0

  • Administrator
  • Hero Member
  • *****
  • Posts: 2743
    • Mark0's Home Page
Re: overcome size limits (USHRT_MAX) in tridscan.py triddefspack.py
« Reply #1 on: February 15, 2024, 11:52:14 PM »
There's a 2KB limit for the fixed pattern area, both practical and "historic" reasons.
It may change in the future, but it would probably part of some major changes.
Sorry but I can't make any promises.