Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Freshclam crash with DatabaseCustomURL for a CVD and also other files, affects versions 1.3.1, 1.3.2, 1.4.0 and 1.4.1 #1364

Open
gotspatel opened this issue Sep 6, 2024 · 7 comments · May be fixed by #1398

Comments

@gotspatel
Copy link

Describe the bug
----------------

I had version 1.3.0 installed and perfectly running on window server 2019 VM, Yesterday tried updating it to 1.4.0 but found issues with freshclam service (it stopped abruptly and immediately on start generating error in eventlog as below)

Faulting application name: freshclam.exe, version: 1.4.0.0, time stamp: 0x66bd0724
Faulting module name: ucrtbase.dll, version: 10.0.17763.6189, time stamp: 0xbc3e3f37
Exception code: 0xc0000005
Fault offset: 0x0000000000025990
Faulting process id: 0x2c68
Faulting application start time: 0x01daff5f791b2503
Faulting application path: C:\Program Files\ClamAV\freshclam.exe
Faulting module path: C:\Windows\System32\ucrtbase.dll
Report Id: 730971f8-e1fc-4528-a917-98a91128208a
Faulting package full name: 
Faulting package-relative application ID: 

Then I tried Fresh Install of Version 1.4.0 Again, but same issue

Later I tried fresh install of 1.4.1, but same issue
So I again tested with 1.3.1, 1.3.2 version also but same issue

I reverted back to 1.3.0 again and there is no problem for freshclam service it works flawlessly as before and updated the signatures

How to reproduce the problem
----------------------------

Try installing it on Windows Server 2019 with all VC Libs installed using abbodi1406 script

I did fresh install on a fresh VM and I was able to reproduce the same for all version 1.3.1, 1.3.2, 1.40. and 1.4.1

C:\Program Files\ClamAV>clamconf -n
Checking configuration files in C:\Program Files\ClamAV

Config file: clamd.conf

LogFile = "C:\Program Files\ClamAV\logs\clamd.log"
LogTime = "yes"
LogVerbose = "yes"
LogRotate = "yes"
ExtendedDetectionInfo = "yes"
TemporaryDirectory = "C:\temp\CLAMTemp"
TCPSocket = "3310"
TCPAddr = "127.0.0.1"
ExcludePath = "C:\Windows", "C:\Scripts"
SelfCheck = "1800"
AlertBrokenExecutables = "yes"
MaxRecursion = "40"

Config file: freshclam.conf

LogTime = "yes"
LogRotate = "yes"
Foreground = "yes"
UpdateLogFile = "C:\Program Files\ClamAV\logs\freshclam.log"
Checks = "24"
DatabaseMirror = "database.clamav.net"
DatabaseCustomURL = <<<< REMOVED AS IT HAS SENSITIVE INFORMATION >>>>

clamav-milter.conf not found

Software settings

Version: 1.3.0
Optional features supported: MEMPOOL AUTOIT_EA06 BZIP2 LIBXML2 PCRE2 JSON RAR

Database information

Database directory: C:\Program Files\ClamAV\database
[3rd Party] badmacro.ndb: 706 sigs
[3rd Party] blurl.ndb: 1953 sigs
[3rd Party] bofhland_cracked_URL.ndb: 40 sigs
[3rd Party] bofhland_malware_attach.hdb: 1836 sigs
[3rd Party] bofhland_malware_URL.ndb: 4 sigs
[3rd Party] bofhland_phishing_URL.ndb: 72 sigs
bytecode.cvd: version 335, sigs: 86, built on Tue Feb 27 21:07:24 2024
daily.cld: version 27387, sigs: 2066357, built on Tue Sep 3 14:08:04 2024
daily.cvd: version 27389, sigs: 2066461, built on Thu Sep 5 14:03:25 2024
[3rd Party] foxhole.ign2: 6 sigs
[3rd Party] foxhole_all.cdb: 149 sigs
[3rd Party] foxhole_all.ndb: 101 sigs
[3rd Party] foxhole_filename.cdb: 3609 sigs
[3rd Party] foxhole_generic.cdb: 215 sigs
[3rd Party] foxhole_js.cdb: 48 sigs
[3rd Party] foxhole_js.ndb: 4 sigs
[3rd Party] foxhole_mail.cdb: 37 sigs
[3rd Party] hackingteam.hsb: 435 sigs
[3rd Party] ignore_list.ign2: 1 sig
[3rd Party] interserver256.hdb: 28766 sigs
[3rd Party] interservertopline.db: 1138 sigs
[3rd Party] javascript.ndb: 10557 sigs
[3rd Party] junk.ndb: 55064 sigs
[3rd Party] jurlbl.ndb: 29699 sigs
[3rd Party] lott.ndb: 2337 sigs
main.cvd: version 62, sigs: 6647427, built on Thu Sep 16 18:02:42 2021
[3rd Party] malware.expert.hdb: 1 sig
[3rd Party] malwarehash.hsb: 1031 sigs
[3rd Party] MiscreantPunch099-Low.ldb: 1199 sigs
[3rd Party] phish.ndb: 30709 sigs
[3rd Party] phishtank.ndb: 1 sig
[3rd Party] porcupine.hsb: 183 sigs
[3rd Party] porcupine.ndb: 1607 sigs
[3rd Party] rogue.hdb: 7287 sigs
[3rd Party] sanesecurity.ftm: 185 sigs
[3rd Party] Sanesecurity_sigtest.yara: 54 sigs
[3rd Party] Sanesecurity_spam.yara: 46 sigs
[3rd Party] scam.ndb: 13097 sigs
[3rd Party] securiteinfo.hdb: 49086 sigs
[3rd Party] securiteinfo.ign2: 222 sigs
[3rd Party] securiteinfoandroid.hdb: 29652 sigs
[3rd Party] securiteinfoascii.hdb: 36181 sigs
[3rd Party] securiteinfohtml.hdb: 32966 sigs
[3rd Party] securiteinfoold.hdb: 4145583 sigs
[3rd Party] securiteinfopdf.hdb: 3408 sigs
[3rd Party] shell.hdb: 4277 sigs
[3rd Party] shell.ldb: 57 sigs
[3rd Party] shellb.db: 292 sigs
[3rd Party] shelter.ldb: 62 sigs
[3rd Party] sigwhitelist.ign2: 18 sigs
[3rd Party] spam.ldb: 2 sigs
[3rd Party] spamattach.hdb: 14 sigs
[3rd Party] spamimg.hdb: 233 sigs
[3rd Party] spam_marketing.ndb: 37626 sigs
[3rd Party] spear.ndb: 1 sig
[3rd Party] spearl.ndb: 1 sig
[3rd Party] urlhaus.ndb: 10705 sigs
[3rd Party] whitelist.fp: 3081 sigs
[3rd Party] winnow.attachments.hdb: 1 sig
[3rd Party] winnow_bad_cw.hdb: 1 sig
[3rd Party] winnow_extended_malware.hdb: 1 sig
[3rd Party] winnow_malware.hdb: 1 sig
[3rd Party] winnow_malware_links.ndb: 133 sigs
[3rd Party] winnow_phish_complete.ndb: 53 sigs
Total number of signatures: 15326165

Platform information

uname: Microsoft Windows Server 6.2 SP0.0 Build 9200
OS: Windows, ARCH: AMD64, CPU: AMD64
zlib version: 1.3.1 (1.3.1), compile flags: 65
platform id: 0x1025c8c80800000000000792

Build information

Microsoft Visual C++: (0.7.146)
sizeof(void*) = 8
Engine flevel: 200, dconf: 200

C:\Program Files\ClamAV>

@gotspatel gotspatel changed the title Fresclam Service Abruptly stops in version 1.3.1, 1.3.2, 1.4.0 and 1.4.1 on windows Server 2019 Freshclam Service Abruptly stops in version 1.3.1, 1.3.2, 1.4.0 and 1.4.1 on windows Server 2019 Sep 6, 2024
@micahsnyder
Copy link
Contributor

Can you confirm if you're the same person to report this issue through Discord?

I haven't heard of any compatibility issues on Windows with 1.3.1 or with Windows versions 8 or newer. So I am very surprised by the issue you're facing.

For ClamAV 1.4.1, 1.3.2, and 1.0.7 I think the will fail with a similar "0xc0000005 application error" on Windows 7.

With ClamAV 1.4.0 and 1.4.1 we provide PDB debugging symbol files, added to try to triage this issue on Windows 7. We didn't solve it, and with no requirement to continue support for Windows 7 we accepted the compatibility issue.

If you want to dig in deeper, you could try starting freshclam.exe from the 1.4.1 or 1.4.0 versions with WinDbg to see if it gives a stack trace or some better explanation for the application error.

@gotspatel
Copy link
Author

YES Same Person

I was even able to replicate the same issues on windows 10 and windows 11 (fresh VM Created for testing Purpose)

I tested and verified same issue on below OS
Server 2019, Server 2016, Windows 10 LTSC, Windows 11 Pro

I will try to use WinDbg and provide further details.

Thanks

@gotspatel
Copy link
Author

Found the Culprit

the conf file I had untill now for 1.3.0 had CRLF and I changed it to LF and it works without any other modifications and without any issue for all versions under windows, especially in freshclam.conf

hope to get a solution from next update onward to allow both CRLF and LF in config files under windows, please

Thanks

@micahsnyder
Copy link
Contributor

@gotspatel that's wild! Let's reopen this issue and rename it. That is absolutely a bug.

@micahsnyder micahsnyder reopened this Sep 17, 2024
@micahsnyder micahsnyder changed the title Freshclam Service Abruptly stops in version 1.3.1, 1.3.2, 1.4.0 and 1.4.1 on windows Server 2019 CR / CRLF line ending compatibility on Windows with versions 1.3.1, 1.3.2, 1.4.0 and 1.4.1 Sep 17, 2024
@gotspatel
Copy link
Author

OK Again I Tried today to install clamAV 1.4.1 on a Fresh VM Windows Server 2019 Standard and the freshclam service is still failing

it crashes with ucrtbase.dll and nt.dll attached the evenviewer details and logs to investigate same happens in the old VM also, (We had reverted to 1.3.0 as it was a production VM and didn't want issues in it)

this VM was also supposed to be production but I wanted to try again, let me know if more details required, I really hope to get it working on windows please, No problem whatsoever with clamd.exe, clamscan.exe, clamdscan.exe

EventViewer.zip

@gotspatel
Copy link
Author

@micahsnyder

I have pinpointed the issues in the freshclam.conf as below, Hope Now you can check what has changed that the versions after 1.3.0 freshclam config doesn't like blank lines and comments in between the urls list

Previously untill 1.3.0 version my freshclam had this EXACTLY IN THIS ORDER and with some blank lines and comment line in between (AND Was and still is WORKING FINE in 1.3.0)

DatabaseCustomURL https://urlhaus.abuse.ch/downloads/urlhaus.ndb
DatabaseCustomURL http://sigs.interserver.net/interserver256.hdb
DatabaseCustomURL http://sigs.interserver.net/interservertopline.db
DatabaseCustomURL http://sigs.interserver.net/shell.ldb
DatabaseCustomURL https://www.securiteinfo.com/get/signatures/<MY_API_KEY>/securiteinfo.hdb
DatabaseCustomURL https://www.securiteinfo.com/get/signatures/<MY_API_KEY>/securiteinfo.ign2
DatabaseCustomURL https://www.securiteinfo.com/get/signatures/<MY_API_KEY>/javascript.ndb
DatabaseCustomURL https://www.securiteinfo.com/get/signatures/<MY_API_KEY>/spam_marketing.ndb
DatabaseCustomURL https://www.securiteinfo.com/get/signatures/<MY_API_KEY>/securiteinfohtml.hdb
DatabaseCustomURL https://www.securiteinfo.com/get/signatures/<MY_API_KEY>/securiteinfoascii.hdb
DatabaseCustomURL https://www.securiteinfo.com/get/signatures/<MY_API_KEY>/securiteinfoandroid.hdb
DatabaseCustomURL https://www.securiteinfo.com/get/signatures/<MY_API_KEY>/securiteinfoold.hdb
DatabaseCustomURL https://www.securiteinfo.com/get/signatures/<MY_API_KEY>/securiteinfopdf.hdb
DatabaseCustomURL http://database.clamav.net/daily.cvd

# http://rbluri.interserver.net/usage.php  http://rbluri.interserver.net/

DatabaseCustomURL http://sigs.interserver.net/interserver256.hdb
DatabaseCustomURL http://sigs.interserver.net/interservertopline.db
DatabaseCustomURL http://sigs.interserver.net/shell.ldb
DatabaseCustomURL http://sigs.interserver.net/whitelist.fp
DatabaseCustomURL http://sigs.interserver.net/shellb.db
DatabaseCustomURL http://sigs.interserver.net/shell.hdb

I changed it as below, removing the blank and comment line from between (ORDER IS NOT IMPORTANT) it works with any order of url but there should not be blank line or comment line in between and it works with 1.3.1, 1.4.0 and 1.4.1

DatabaseCustomURL http://database.clamav.net/daily.cvd
DatabaseCustomURL http://sigs.interserver.net/shellb.db
DatabaseCustomURL http://sigs.interserver.net/shell.ldb
DatabaseCustomURL http://sigs.interserver.net/shell.hdb
DatabaseCustomURL http://sigs.interserver.net/whitelist.fp
DatabaseCustomURL https://urlhaus.abuse.ch/downloads/urlhaus.ndb
DatabaseCustomURL http://sigs.interserver.net/interserver256.hdb
DatabaseCustomURL http://sigs.interserver.net/interservertopline.db
DatabaseCustomURL https://www.securiteinfo.com/get/signatures/<MY_API_KEY>/javascript.ndb
DatabaseCustomURL https://www.securiteinfo.com/get/signatures/<MY_API_KEY>/securiteinfo.hdb
DatabaseCustomURL https://www.securiteinfo.com/get/signatures/<MY_API_KEY>/securiteinfo.ign2
DatabaseCustomURL https://www.securiteinfo.com/get/signatures/<MY_API_KEY>/spam_marketing.ndb
DatabaseCustomURL https://www.securiteinfo.com/get/signatures/<MY_API_KEY>/securiteinfoold.hdb
DatabaseCustomURL https://www.securiteinfo.com/get/signatures/<MY_API_KEY>/securiteinfopdf.hdb
DatabaseCustomURL https://www.securiteinfo.com/get/signatures/<MY_API_KEY>/securiteinfohtml.hdb
DatabaseCustomURL https://www.securiteinfo.com/get/signatures/<MY_API_KEY>/securiteinfoascii.hdb
DatabaseCustomURL https://www.securiteinfo.com/get/signatures/<MY_API_KEY>/securiteinfoandroid.hdb

@micahsnyder
Copy link
Contributor

It is not a CRLF issue.

I was able to reproduce the issue with this smaller config:

DatabaseMirror http://localhost:8000
DatabaseCustomURL http://localhost:8000/extra.wdb
DatabaseCustomURL http://localhost:8000/daily.cvd

I'm hosting databases on the same system with port 8000 so as not to rate limit myself.

I'll have a fix for it shortly.

micahsnyder added a commit to micahsnyder/clamav-micah that referenced this issue Oct 30, 2024
…iles

Freshclam may crash if using DatabaseCustomURL for a CVD and multiple
other files. The issue occurs because of a bad index in the "do not
prune" list.

Fixes: Cisco-Talos#1364
@micahsnyder micahsnyder changed the title CR / CRLF line ending compatibility on Windows with versions 1.3.1, 1.3.2, 1.4.0 and 1.4.1 Freshclam crash with DatabaseCustomURL for a CVD and also other files, affects versions 1.3.1, 1.3.2, 1.4.0 and 1.4.1 Oct 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants