update to 1.3 cross-platform

README.linux-compilation.md

@@ -0,0 +1,41 @@
+# Compiling instruction for _FastFinder_ on Linux
+
+_FastFinder_ was originally designed for Windows platform but it also work perfectly on Linux. Unlike  other Go programs, if you want to compile or run it from source, you will need to install some libraries and compilation tools. Indeed, _FastFinder_ is strongly dependent of libyara, go-yara and CGO. Here's a little step by step guide: 
+
+## Before installation
+
+Please ensure having:
+* Go >= 1.17
+* GOPATH / GOOS / GOARCH correctly set 
+* administrator rights to insall 
+
+## Compile YARA
+
+1/ download YARA latest release source tarball (https://github.com/VirusTotal/yara)
+2/ Make sure you have `automake`, `libtool`, `make`, `gcc` and `pkg-config` installed in your system. 
+2/ unzip and compile yara like this: 
+```
+tar -zxf yara-<version>.tar.gz
+cd <version>.
+./bootstrap.sh
+./configure
+make
+make install
+```
+3/ Run the test cases to make sure that everything is fine:
+```
+make check
+```
+
+## Configure CGO
+CGO will link libyara and compile C instructions used by _Fastfinder_ (through go-yara project). Compiler and linker flags have to be set via the CGO_CFLAGS and CGO_LDFLAGS environment variables like this:
+```
+export CGO_CFLAGS="-I<YARA_SRC_PATH>/libyara/include"
+export CGO_LDFLAGS="-L<YARA_SRC_PATH>/libyara/.libs -lyara"
+```
+
+## You're ready to Go!
+You can compile _FastFinder_ with the following command:
+```
+go build -tags yara_static -a -ldflags '-s -w' .
+```

README.md

@@ -1,31 +1,38 @@
 # _FastFinder_ - Incident Response - Fast suspicious file finder
+[![Golang](https://img.shields.io/badge/Go-1.17-blue.svg)](https://golang.org)
+![Linux](https://img.shields.io/badge/Supports-Linux-green.svg)
+![windows](https://img.shields.io/badge/Supports-windows-green.svg)
 
 ## What is this project designed for?
-_FastFinder_ is a lightweight tool made for threat hunting, live forensics and triage on Windows Platform. It is 
+_FastFinder_ is a lightweight tool made for threat hunting, live forensics and triage on both Windows and Linux Platforms. It is 
 focused on enpoint enumeration and suspicious file finding based on various criterias:
 * file path / name
+* md5 / sha1 / sha256 checksum
 * simple string content match
 * complex content condition(s) based on YARA
 
 ### Installation 
 Compiled release of this software are available. If you want to compile 
 from sources, it could be a little bit tricky because it strongly depends of 
-_go-yara_ and CGO compilation. Anyway, you'll find a detailed documentation [here](README.windows-compilation.md)
+_go-yara_ and CGO compilation. Anyway, you'll find a detailed documentation [for windows](README.windows-compilation.md) and for [for linux](README.linux-compilation.md)
 
 ### Usage 
 ```
-fastfinder [-h|--help] -c|--configuration "<value>" [-b|--build
-                  "<value>"] [-o|--output "<value>"] [-n|--nowindow]
+usage: fastfinder [-h|--help] [-c|--configuration "configuration.yaml"] [-b|--build
+                  "path_to_package_bin"] [-o|--output "log_file.log"] [-n|--nowindow]
+                  [-p|--showprogress] [-v|--version]
 
                   Incident Response - Fast suspicious file finder
 
 Arguments:
 
   -h  --help           Print help information
-  -c  --configuration  Fastfind configuration file
+  -c  --configuration  Fastfind configuration file. Default: configuration.yaml
   -b  --build          Output a standalone package with configuration and rules in a single binary
   -o  --output         Save fastfinder logs in the specified file
   -n  --nowindow       Hide fastfinder window
+  -p  --showprogress   Display I/O analysis progress
+  -v  --version        Display fastfinder version
 ``` 
 
 Depending on where you are looking for files, _FastFinder_ could be used with admin OR simple user rights. 
@@ -57,8 +64,14 @@ output:
 * input content grep strings are always case SENSITIVE
 * backslashes haven't to be escaped on simple string pattern (see example) 
 
-## About this project and future versions
+## About this project
 I initially created this project to automate the creation of fastfind on a wide computer network. 
 It fulfills the needs I have today, nevertheless if you have complementary ideas, do not hesitate 
 to ask for, I will see to implement them if they can be useful for everyone.
 On the other hand, pull request will be studied carefully.
+
+## Future releases
+I don't plan to add any additional features right now. The next release will be focused on:
+* Stability / performance improvements
+* Unit testing / Code testing coverage / CI
+* Build more examples based on live malwares tradecraft and threat actor campaigns

README.windows-compilation.md

@@ -1,7 +1,6 @@
+# Compiling instruction for _FastFinder_ on Windows
 
-# Installing _FastFinder_ on Windows
-
-_FastFinder_ is design for Windows platform but it's a little bit tricky because it's strongly dependant of go-yara and CGO. Here's a little step by step guide: 
+_FastFinder_ was originally designed for Windows platform but it's a little bit tricky to compile because it's strongly dependant of go-yara and CGO. Here's a little step by step guide: 
 
 ## Before installation
 

examples/example_configuration_linux.yaml

@@ -0,0 +1,19 @@
+input:
+    path: []
+    content:
+        grep: []
+        yara:
+          - './examples/example_rule_linux.yar'
+        checksum:
+         - 'bf1cde9c94c301cdc3b5486f2f3fe66b'
+         - '41ba1bd49cb22466e422098d184bd4267ef9529e'
+         - 'e875b1185577ff872fbaabde481cc196af03745c530403c8303f00fe35859bf7'
+options:
+    contentMatchDependsOnPathMatch: false
+    findInHardDrives: true
+    findInRemovableDrives: false
+    findInNetworkDrives: false
+    findInCDRomDrives: false
+output:
+    base64Files: true
+    filesCopyPath: ''

examples/example_rule_linux.yar

@@ -0,0 +1,14 @@
+rule fastfinder_example{
+	meta:
+		name = "fastfinder_example"
+		description = "Example of fastfinder yara match (on legitimate linux 'more' binary)"
+		reference = "https://github.com/codeyourweb/fastfinder"
+	strings:
+		$str1 = "GNU"
+		$str3 = "--More--"
+		$str4 = "file perusal filter for CRT viewing"
+		$str5 = "Press 'h' for instructions"
+		$op = { ba 05 00 00 00 31 ff 4? 8d 35 ?? ?? ?? ?? e8 ?? ?? ?? ?? 4? 89 ee 4? 89 c7 e8 ?? ?? ?? ?? ba 05 00 00 00 31 ff 4? 8d 35 ?? ?? ?? ?? e8 ?? ?? ?? ??}
+	condition:
+		all of them and uint16(0) == 0x457f
+}

finder.go

@@ -14,9 +14,11 @@ import (
 
 // PathsFinder try to match regular expressions in file paths slice
 func PathsFinder(files *[]string, patterns []*regexp2.Regexp) *[]string {
+	InitProgressbar(int64(len(*files)))
 	var matchingFiles []string
 	for _, expression := range patterns {
 		for _, f := range *files {
+			ProgressBarStep()
 			if match, _ := expression.MatchString(f); match {
 				matchingFiles = append(matchingFiles, f)
 			}
@@ -29,7 +31,9 @@ func PathsFinder(files *[]string, patterns []*regexp2.Regexp) *[]string {
 // FindInFiles check for pattern or checksum match in files slice
 func FindInFiles(files *[]string, patterns []string, checksum []string) *[]string {
 	var matchingFiles []string
+	InitProgressbar(int64(len(*files)))
 	for _, f := range *files {
+		ProgressBarStep()
 		b, err := ioutil.ReadFile(f)
 		if err != nil {
 			LogMessage(LOG_ERROR, "[ERROR]", "Unable to read file", f)

go.mod

@@ -7,8 +7,19 @@ require (
 	github.com/gen2brain/go-unarr v0.1.2
 	github.com/h2non/filetype v1.1.3
 	github.com/hillu/go-yara/v4 v4.1.0
-	golang.org/x/sys v0.0.0-20211124211545-fe61309f8881
+	golang.org/x/sys v0.0.0-20211205182925-97ca703d548d
 	gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b
 )
 
-require github.com/dlclark/regexp2 v1.4.0
+require (
+	github.com/dlclark/regexp2 v1.4.0
+	github.com/schollz/progressbar/v3 v3.8.3
+)
+
+require (
+	github.com/mattn/go-runewidth v0.0.13 // indirect
+	github.com/mitchellh/colorstring v0.0.0-20190213212951-d06e56a500db // indirect
+	github.com/rivo/uniseg v0.2.0 // indirect
+	golang.org/x/crypto v0.0.0-20211202192323-5770296d904e // indirect
+	golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 // indirect
+)

go.sum

@@ -1,5 +1,8 @@
 github.com/akamensky/argparse v1.3.1 h1:kP6+OyvR0fuBH6UhbE6yh/nskrDEIQgEA1SUXDPjx4g=
 github.com/akamensky/argparse v1.3.1/go.mod h1:S5kwC7IuDcEr5VeXtGPRVZ5o/FdhcMlQz4IZQuw64xA=
+github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
+github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/dlclark/regexp2 v1.4.0 h1:F1rxgk7p4uKjwIQxBs9oAXe5CqrXlCduYEJvrF4u93E=
 github.com/dlclark/regexp2 v1.4.0/go.mod h1:2pZnwuY/m+8K6iRw6wQdMtk+rH5tNGR1i55kozfMjCc=
 github.com/gen2brain/go-unarr v0.1.2 h1:17kYZ2WMCVFrnmU4A+7BeFXblIOyE8weqggjay+kVIU=
@@ -8,8 +11,40 @@ github.com/h2non/filetype v1.1.3 h1:FKkx9QbD7HR/zjK1Ia5XiBsq9zdLi5Kf3zGyFTAFkGg=
 github.com/h2non/filetype v1.1.3/go.mod h1:319b3zT68BvV+WRj7cwy856M2ehB3HqNOt6sy1HndBY=
 github.com/hillu/go-yara/v4 v4.1.0 h1:ZLT9ar+g5r1IgEp1QVYpdqYCgKMNm7DuZYUJpHZ3yUI=
 github.com/hillu/go-yara/v4 v4.1.0/go.mod h1:rkb/gSAoO8qcmj+pv6fDZN4tOa3N7R+qqGlEkzT4iys=
-golang.org/x/sys v0.0.0-20211124211545-fe61309f8881 h1:TyHqChC80pFkXWraUUf6RuB5IqFdQieMLwwCJokV2pc=
-golang.org/x/sys v0.0.0-20211124211545-fe61309f8881/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+github.com/k0kubun/go-ansi v0.0.0-20180517002512-3bf9e2903213/go.mod h1:vNUNkEQ1e29fT/6vq2aBdFsgNPmy8qMdSay1npru+Sw=
+github.com/mattn/go-isatty v0.0.14/go.mod h1:7GGIvUiUoEMVVmxf/4nioHXj79iQHKdU27kJ6hsGG94=
+github.com/mattn/go-runewidth v0.0.13 h1:lTGmDsbAYt5DmK6OnoV7EuIF1wEIFAcxld6ypU4OSgU=
+github.com/mattn/go-runewidth v0.0.13/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=
+github.com/mitchellh/colorstring v0.0.0-20190213212951-d06e56a500db h1:62I3jR2EmQ4l5rM/4FEfDWcRD+abF5XlKShorW5LRoQ=
+github.com/mitchellh/colorstring v0.0.0-20190213212951-d06e56a500db/go.mod h1:l0dey0ia/Uv7NcFFVbCLtqEBQbrT4OCwCSKTEv6enCw=
+github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
+github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
+github.com/rivo/uniseg v0.2.0 h1:S1pD9weZBuJdFmowNwbpi7BJ8TNftyUImj/0WQi72jY=
+github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
+github.com/schollz/progressbar/v3 v3.8.3 h1:FnLGl3ewlDUP+YdSwveXBaXs053Mem/du+wr7XSYKl8=
+github.com/schollz/progressbar/v3 v3.8.3/go.mod h1:pWnVCjSBZsT2X3nx9HfRdnCDrpbevliMeoEVhStwHko=
+github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
+github.com/stretchr/testify v1.3.0 h1:TivCn/peBQ7UY8ooIcPgZFpTNSz0Q2U6UrFlUfqbe0Q=
+github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
+golang.org/x/crypto v0.0.0-20210817164053-32db794688a5/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
+golang.org/x/crypto v0.0.0-20211202192323-5770296d904e h1:MUP6MR3rJ7Gk9LEia0LP2ytiH6MuCfs7qYz+47jGdD8=
+golang.org/x/crypto v0.0.0-20211202192323-5770296d904e/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
+golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
+golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
+golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20210910150752-751e447fb3d0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20211205182925-97ca703d548d h1:FjkYO/PPp4Wi0EAUOVLxePm7qVW4r4ctbWpURyuOD0E=
+golang.org/x/sys v0.0.0-20211205182925-97ca703d548d/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
+golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
+golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 h1:JGgROgKl9N8DuW20oFS5gxc+lE67/N3FcwmBPMe7ArY=
+golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
+golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
+golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
+golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
 gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b h1:h8qDotaEPuJATrMmW04NCwg7v22aHH28wwpauUhK9Oo=

logger.go

@@ -43,7 +43,7 @@ func StdoutToLogFile(outLogPath string) {
 		scanner := bufio.NewScanner(rd)
 		for scanner.Scan() {
 			stdoutLine := scanner.Text()
-			multiWriter.Write([]byte(stdoutLine + "\r\n"))
+			multiWriter.Write([]byte(stdoutLine + LineBreak))
 		}
 	}()
 }
@@ -68,7 +68,7 @@ func StderrToLogFile(outLogPath string) {
 		scanner := bufio.NewScanner(rd)
 		for scanner.Scan() {
 			stdoutLine := scanner.Text()
-			multiWriter.Write([]byte(stdoutLine + "\r\n"))
+			multiWriter.Write([]byte(stdoutLine + LineBreak))
 		}
 	}()
 }