Feature/symfony upgrade to.3.4 (#231)

* fix install of grunt-cli to only install the 1.3.2 so that it does not brake with older versions of node

* Renew SSL certs until 2026

* Removed symfony/security-csrf package since it was colliding with symfony/security which replaces thr former one. Thanks to that, symfony/security can be safely updated to 2.8.52 .

* Removed enforcement to keep some Symfony package in v2.8.0 so they are now updatable to 2.8.52

* Added (un)zip packages to Dockerfile so we remove Composer warning (#170)

* Bug/fix unit tests (#171)

* Added missing PDO dependency which is in fact needed

* Fixed Sponsor model unit tests

* Removed pointless comments and deprecated method on Mail model

* Fixed Mail model unit tests

* Fixed TextTest translations tests due to missing and/or incorrect HTML tags/attributes

* Updated Matcher model tests

* Enforce PHP 7.1 as minimum PHP version in Composer

* Fixed HTML structure on newsletter template and removed commented CSS classes

* Improved readability, added parameter types and some small refactors on Mail model

* Improved readability and added parameter types on MatcherTest class

* Further cleanup on User and MatcherTest classes

* Downgraded minimum PHP version to the production one

* Reintroduced sqlWhere variable so we don't get errors since it's used in the code

* Fully reintroduced the Sponsor 'type' property and fixed related failing test

* Refactored call to obtain Sponsors list into a method

* Fix SQL query to obtain Sponsors list

* Fix Mail model tests along with issues on the HTML formatting with 2 related templates

* Changed method syntax so is PHP 7.0 compliant (it was using PHP 7.1 syntax)

* Fix Matcher model tests (some translations were missing when querying for them)

* Reduced tests verbosity (we don't need to know if users, projects, etc were removed when running tests)

* Ignored .idea (Jetbrains IDEs) folder

* Bumped PHPMailer to 6.x and fixed Mail model and test with related changes

* Bumped Mockery to 1.x

* Added OpenSSL extension, since it's used by AmazonSns and AuthListener classes

* Updated amazon-s3-php-class with many changes (last updated on 2015)

* Assure PHPMailer minimum version is 6.4

* Updated to PHPUnit 6.5

* Removed empty test along with unused imports

* Moved PHP extensions on composer file right after the PHP version for easy spotting

* Reduced verbosity on tests output so we can focus on seeing if the focus passed, failed, errored or were skipped

* Remove deprecated asserts and upgrade PHPUnit from 6.x to 7.x

* Bumped min PHP version to 7.1

* Upgrade PHP from 7.0/7.1 to 7.4

* Upgraded PHPUnit to 8.x (depends on PR that enforces PHP 7.4)

* Removed deprecated expectedException (will be removed on PHPUnit 9.x). Flipped assertEquals comparators to be compliant with expected/actual order.

* Removed deprecated class

* Got rid of last array curly braces access

* Removed duplicated key

* Removed deprecated (and unused) FlattenException import

* Removed last noisy message on PHPUnit so we only have test results status

* WIP Upgrade to Symfony 3.x

* Tidy index(_env).php files so they have less differences between them, for a future merge

* Got rid of index_dev.php in favor of index.php that read ENV DEBUG parameter

* Added import, rewritten some PHPDocs and added some return types

* Clean container.php

* Refactor getName() method from custom Symfony Form Types, so they just have getBlockPrefix() method

* Refactor to add Form Types as is required in Symfony 3

* Fix install of migrations on PHP 7.4 and latest Composer dependencies

* Fix RouteListener initialization. Fix DB setup.

* Use ::class to register services in the ServiceContainer instead of strings

* Fix composer.json so it reckognizes goteo-private tests

* Fix tests for App class (due to having renamed a property)

* Updated Omnipay dependency to the new one from league

* Grouped Composer dependencies

* Used Symfony HttpKernel class instead of HttpKernel\HttpKernel

* Removed unneeded package

* Cleanup Omnipay classes and fix some issues while initilizing the Payment Gateway

* Made final changes to support all Omnipay payment methods using Omnipay v3

* Deprecated static AbstractPaymentMethod::getId() method in favor of non static one

* Converted static calculateComission method into non-static calculateCommission

* Converted static AbstractPaymentMethod::isInternal() into a non-static method

* Add parameter / return types and import used classes

* Updated TelegramBot so it doesn't use deprecated code

* Reviewed controllers to ensure they follow Symfony 3 convention

* Add User property type

* Removed unused code

* Tidy up classes

* Removed unused parameters from PHPDoc

* Upgraded Symfony from 3.0 to 3.1

* Included some used classes

* Finalize migration to symfony/forms 3.x (there must be a getBlockPrefix() method on a new Type, even if their parents have the method

* Enforce getModelForm() & createFormBuilder() from controllers to return FormProcessorInterface & FormFactory respectively

* Fix return parameter of Controller::createFormBuilder() method and the controllers that inherit it

* Upgraded to Symfony 3.2

* Symfony packages upgraded to 3.3

* WIP Migration to Symfony 3.4 almost completed

* Fix call to Mail::renderEmailTemplate (got renamed recently)

* Fix Settings Dashboard form (ChoiceType elements had flipped key / values)

* Fix loading of ChoiceType and some minor refactors

* Fix ChoiceType for some forms related to Contract PersonDocumentTypes

* Revert BooleanType::getBlockPrefix() method so it returns 'boolean' which to match it's class name

* Refactor ProjectOverviewForm to fix it and to ease its reading

* Fix import of PDOException (it was written incorrectly)

* Fix ChoiceTypes on several Project related forms

* Refactor to use the new Controller:getModelForm() method that requires a class instead of just a string. Fixed some Symfony Forms due to incomplete migration when Symfony 3.0 upgrade.

* Refactored last controller that used the deprecated method to guess SymfonyForm class

* Tiny refactors

* Removed unnecessary EventDispatcher from Console class

* Clean up of some classes

* Change test to testLegacy, since the deprecation notice is due to Symfony code (and is due to expire on Symfony 4.3)

* Fix some Symfony Forms with unfinished migration to Symfony 3.x. Remove some unused code / comments.

* General cleanup of controllers

* Tiny cleanup

* Fix access to some admin pages

* Fix usage of DatepickerType so it actually replaces the Symfony type while keeping all it's parents features

* Fix admin blog add/edit form

* Fix behaviour of some custom Symfony Form Types

* Fix FilterForm so it conforms to Symfony 3.x standards

* Readd workshop_location table since it got removed sometime ago by mistake

* Fix Symfony Console (probably due to some incomplete step after migrating to Symfony 3.x)

* Fix admin workshop add/edit form

* Fix some translations

* Refactor Sponsors controller

* Fix broken CSS on Call edit form for rewards items

* Fix admin Stories add/edit form

* Fix CSS error while displaying TypeaheadType form elements

* Fix Controller::redirect() parameter types

* Fix admin Channel Questionnaire and Channel Section forms to add/edit

* Fix Admin Charts API controller

* Cleanup SessionListener

* Reformatted Filter and Project models

* Simplify public/index.php

* Cleanup main PHP classes

* Minor refactors on controllers

* Removed deprecated mcrypt PHP extension

* Removed deprecated mcrypt PHP extension from composer.lock

* Fix SQL migration

* Fix SQL migration related to mail table

* Fix SQL migration related to mail table

* Fix SQL migration related to mail table

* Improved Docker readme

* make loggin resilient to Gelf updates failures (#225)

* make loggin resilient to Gelf updates failures

* ignore redundant exception thrown

* fix on deprecated implode call (#229)

* update oauth library for linkedin (#226)

* add deployer script

* refactor subdomain processing and add test to it (#205)

* fix/var subs (#215)

* refactor subdomain processing and add test to it (#204)

* refactor subdomain processing and add test to it

* add other envs to deployment

* make config process deep ENV substitution

* fix/var subs (#216)

* refactor subdomain processing and add test to it (#204)

* refactor subdomain processing and add test to it

* add other envs to deployment

* make config process deep ENV substitution

* fix namespace

* update oauth library for linkedin

* update up to a working commit

* fix linkedin call

* remove orphan location stats (#230)

* Moved Omnipay/Mock/Gateway class to goteo repository (is needed to run PHPUnits successfully)

Co-authored-by: Javier <javicarrillo83@gmail.com>
Co-authored-by: David Igón <david@goteo.org>
Co-authored-by: Ivan Vergés <ivan@platoniq.net>
Co-authored-by: David <david.igon@riseup.net>

Resources/templates/responsive/channel/call/partials/map.php

@@ -21,7 +21,7 @@
   }
 
   if ($map_config['center']) {
-    $url .= '/' . implode($map_config['center'],',');
+    $url .= '/' . implode(',', $map_config['center']);
   }
 
 

composer.json

@@ -18,7 +18,7 @@
         "jdorn/file-system-cache": "dev-master",
         "intervention/image": "~2.0",
         "mibe/feedwriter": "~1.0",
-        "lusitanian/oauth": "^0.8.9",
+        "lusitanian/oauth": "dev-master#c3b5e51fcf86ebe83023d34744d7da131a9744a5",
         "iignatov/lightopenid": "~1.0",
         "paypal/adaptivepayments-sdk-php": "*",
         "paypal/merchant-sdk-php": "*",

extend/extra-logger/start.php

@@ -24,7 +24,6 @@
 $gelf = Config::get('plugins.extra-logger.gelf');
 if($gelf && $gelf["host"]) {
     $handler = new GelfHandler(new Publisher( new UdpTransport($gelf["host"], $gelf["port"]) ));
-
     App::getService('logger')->pushHandler($handler);
     App::getService('syslogger')->pushHandler($handler);
     App::getService('paylogger')->pushHandler($handler);

src/Goteo/Application/EventListener/ExceptionListener.php

@@ -97,17 +97,22 @@ static function jTraceEx($e, $with_message = true, $seen = null) {
     protected function logException(\Exception $exception, $message) {
         if (null !== $this->logger) {
             $message = str_replace(["\n", "\r"],[" ", ""], $message);
-            if ($exception instanceof LegacyError) {
-                $this->logger->warning('Kernel Exception', ['etype' => 'LegacyError', 'trace' => self::jTraceEx($exception, false), 'exception' => $exception, 'message' => $message]);
-            } elseif (!$exception instanceof HttpExceptionInterface || $exception->getStatusCode() >= 500) {
-                $this->logger->critical('Kernel Exception', ['etype' => 'HttpException', 'trace' => self::jTraceEx($exception, false), 'exception' => $exception, 'message' => $message]);
-            } elseif ($exception instanceof ModelNotFoundException) {
-                $this->logger->warning('Kernel Exception', ['etype' => 'NotFound', 'trace' => self::jTraceEx($exception, false), 'exception' => $exception, 'message' => $message]);
-            } elseif ($exception instanceof ControllerAccessDeniedException) {
-                $this->logger->warning('Kernel Exception', ['etype' => 'AccessDenied', 'trace' => self::jTraceEx($exception, false), 'exception' => $exception, 'message' => $message]);
-            } else {
-                $this->logger->error('Kernel Exception', ['etype' => 'Exception', 'trace' => self::jTraceEx($exception, false), 'exception' => $exception, 'message' => $message]);
+            try {
+                if ($exception instanceof LegacyError) {
+                    $this->logger->warning('Kernel Exception', ['etype' => 'LegacyError', 'trace' => self::jTraceEx($exception, false), 'exception' => $exception, 'message' => $message]);
+                } elseif (!$exception instanceof HttpExceptionInterface || $exception->getStatusCode() >= 500) {
+                    $this->logger->critical('Kernel Exception', ['etype' => 'HttpException', 'trace' => self::jTraceEx($exception, false), 'exception' => $exception, 'message' => $message]);
+                } elseif ($exception instanceof ModelNotFoundException) {
+                    $this->logger->warning('Kernel Exception', ['etype' => 'NotFound', 'trace' => self::jTraceEx($exception, false), 'exception' => $exception, 'message' => $message]);
+                } elseif ($exception instanceof ControllerAccessDeniedException) {
+                    $this->logger->warning('Kernel Exception', ['etype' => 'AccessDenied', 'trace' => self::jTraceEx($exception, false), 'exception' => $exception, 'message' => $message]);
+                } else {
+                    $this->logger->error('Kernel Exception', ['etype' => 'Exception', 'trace' => self::jTraceEx($exception, false), 'exception' => $exception, 'message' => $message]);
+                }
+            } catch(\RuntimeException $e) {
+                // Ignore error logging here, it should be cached by the app
             }
+
         }
     }
 

src/Goteo/Console/Command/DBVerifierCommand.php

@@ -10,11 +10,14 @@
 
 namespace Goteo\Console\Command;
 
+use Exception;
 use Goteo\Application\Config;
 use Goteo\Model\Mail;
 use Goteo\Model\Project;
 use Goteo\Model\Template;
 use Goteo\Util\AnsiConverter\Theme\SolarizedLightTheme;
+use PDO;
+use RuntimeException;
 use SensioLabs\AnsiConverter\AnsiToHtmlConverter;
 use Symfony\Component\Console\Input\ArrayInput;
 use Symfony\Component\Console\Input\InputArgument;
@@ -81,7 +84,7 @@ protected function execute(InputInterface $input, OutputInterface $output)
         $update = $input->getOption('update');
         $scope = $input->getArgument('scope');
         if(!in_array($scope, ['all', 'feed', 'mailing', 'token', 'blocked', 'toolkit', 'predict'])) {
-            throw new \Exception('Scope is not valid!');
+            throw new Exception('Scope is not valid!');
         }
 
         $verbose = $output->isVerbose();
@@ -90,7 +93,7 @@ protected function execute(InputInterface $input, OutputInterface $output)
             $days = (int) $input->getOption('days');
             if($days == -1) $days = 120;
             if($days < 30) {
-                throw new \Exception('Number of days must be greater than 30!');
+                throw new Exception('Number of days must be greater than 30!');
             }
             $index = $fixes = 0;
             $output->writeln("Checking old feed data...");
@@ -101,7 +104,7 @@ protected function execute(InputInterface $input, OutputInterface $output)
             if($verbose) {
                 $query = Project::query("SELECT * FROM feed $where");
 
-                foreach ($query->fetchAll(\PDO::FETCH_CLASS) as $feed) {
+                foreach ($query->fetchAll(PDO::FETCH_CLASS) as $feed) {
                     $output->writeln("Found old feed <info>{$feed->title}</info> with ID <comment>{$feed->id}</comment> and date <comment>{$feed->datetime}</comment>");
                 }
             }
@@ -122,14 +125,14 @@ protected function execute(InputInterface $input, OutputInterface $output)
             $days = (int) $input->getOption('days');
             if($days == -1) $days = 4;
             if($days < 1) {
-                throw new \Exception('Number of days must be greater than 2!');
+                throw new Exception('Number of days must be greater than 2!');
             }
 
             $output->writeln("Checking token older thant $days days...");
             // eliminamos los tokens que tengan más de $days días
             $sql = "SELECT id, token FROM user WHERE token IS NOT NULL AND token != '' AND token LIKE '%¬%'";
             $query = Project::query($sql);
-            foreach ($query->fetchAll(\PDO::FETCH_OBJ) as $row) {
+            foreach ($query->fetchAll(PDO::FETCH_OBJ) as $row) {
                 $parts = explode('¬', $row->token);
                 $datepart = strtotime($parts[2]);
                 $today = date('Y-m-d');
@@ -154,7 +157,7 @@ protected function execute(InputInterface $input, OutputInterface $output)
             $days = (int) $input->getOption('days');
             if($days == -1) $days = 365;
             if($days < 30) {
-                throw new \Exception('Number of days must be greater than 30!');
+                throw new Exception('Number of days must be greater than 30!');
             }
 
             $output->writeln("Checking old mail data...");
@@ -164,7 +167,7 @@ protected function execute(InputInterface $input, OutputInterface $output)
             $query = Project::query("SELECT * FROM mail $where", [':template1' => Template::NEWSLETTER, ':template2' => Template::MESSAGE_DONORS]);
 
             $found = 0;
-            foreach ($query->fetchAll(\PDO::FETCH_CLASS) as $mail) {
+            foreach ($query->fetchAll(PDO::FETCH_CLASS) as $mail) {
                 if($verbose) {
                     $output->writeln("Found old mail <info>{$mail->email}</info>, Template <comment>{$mail->template}</comment> Subject <info>[{$mail->subject}]</info> with ID <comment>{$mail->id}</comment> and date <comment>{$mail->date}</comment>");
                 }
@@ -177,6 +180,9 @@ protected function execute(InputInterface $input, OutputInterface $output)
                     }
                 }
             }
+            if($update) {
+                Project::query("DELETE FROM mail_stats_location WHERE id NOT IN (SELECT id FROM mail_stats)");
+            }
             $query = Project::query("SELECT count(*) as total FROM mail");
             $total = $query->fetchColumn();
             $output->writeln("Found <comment>$found</comment> <info>mail records</info> older than <comment>$days days</comment> from a total of <comment>$total</comment> records");
@@ -187,7 +193,7 @@ protected function execute(InputInterface $input, OutputInterface $output)
             $days = (int) $input->getOption('days');
             if($days == -1) $days = 120;
             if($days < 30) {
-                throw new \Exception('Number of days must be greater than 120!');
+                throw new Exception('Number of days must be greater than 120!');
             }
 
             $output->writeln("Checking old blocked users for mailing...");
@@ -196,7 +202,7 @@ protected function execute(InputInterface $input, OutputInterface $output)
             $query = Project::query("SELECT * FROM mailer_control $where");
 
             $found = 0;
-            foreach ($query->fetchAll(\PDO::FETCH_CLASS) as $usr) {
+            foreach ($query->fetchAll(PDO::FETCH_CLASS) as $usr) {
                 if($verbose) {
                     $output->writeln("Found blocked subscriber <info>{$usr->email}</info>, Template <comment>{$usr->bounces}</comment> Reason <info>[{$usr->last_reason}]</info> and date <comment>{$usr->modified}</comment>");
                 }
@@ -246,7 +252,7 @@ protected function execute(InputInterface $input, OutputInterface $output)
                 $mailer = Mail::createFromHtml(Config::getMail('fail'), '', "DATABASE INCONSISTENCY in [" .Config::get('url.main')."]", $html);
                 $errors = [];
                 if(!$mailer->send($errors)) {
-                    throw new \RuntimeException('Error sending email: ' . implode("\n", $errors));
+                    throw new RuntimeException('Error sending email: ' . implode("\n", $errors));
                 }
             } else {
                 $output->writeln("<info>Everything ok</info>");
@@ -272,7 +278,7 @@ protected function execute(InputInterface $input, OutputInterface $output)
                 $mailer = Mail::createFromHtml(Config::getMail('fail'), '', "FAILING PROJECTS PREDICTION in [" .Config::get('url.main')."]", $html);
                 $errors = [];
                 if(!$mailer->send($errors)) {
-                    throw new \RuntimeException('Error sending email: ' . implode("\n", $errors));
+                    throw new RuntimeException('Error sending email: ' . implode("\n", $errors));
                 }
             } else {
                 $output->writeln("<info>Everything ok</info>");

src/Goteo/Core/Traits/LoggerTrait.php

@@ -11,10 +11,8 @@
 namespace Goteo\Core\Traits;
 
 use Goteo\Application\App;
-use Goteo\Application\Config;
 use Goteo\Util\Monolog\Processor\WebProcessor;
 use Psr\Log\LoggerInterface;
-use RuntimeException;
 
 /**
  * Trait to add log common methods
@@ -37,15 +35,7 @@ public function getLog() {
     public function log($message, array $context = [], $func = 'info') {
         $logger = $this->getLog();
         if (null !== $logger && method_exists($logger, $func)) {
-            if(Config::get('debug')) {
-                return $logger->$func($message, WebProcessor::processObject($context));
-            } else {
-                try {
-                    return $logger->$func($message, WebProcessor::processObject($context));
-                } catch(RuntimeException $e) {
-                    // nothing here, if not in debug mode, failure to process logs is ignored
-                }
-            }
+            return $logger->$func($message, WebProcessor::processObject($context));
         }
     }
 

src/Goteo/Core/Traits/StaticLoggerTrait.php

@@ -12,8 +12,6 @@
 
 use Goteo\Util\Monolog\Processor\WebProcessor;
 use Psr\Log\LoggerInterface;
-use Goteo\Application\Config;
-use RuntimeException;
 
 /**
  * Trait to use log on legacy classes
@@ -32,15 +30,7 @@ static public function getLogger(LoggerInterface $logger) {
 
     static public function log($message, array $context = [], $func = 'info') {
         if(static::$logger) {
-            if(Config::get('debug')) {
-                return static::$logger->$func($message, WebProcessor::processObject($context));
-            } else {
-                try {
-                    return static::$logger->$func($message, WebProcessor::processObject($context));
-                } catch(RuntimeException $e) {
-                    // nothing here, if not in debug mode, failure to process logs is ignored
-                }
-            }
+            return static::$logger->$func($message, WebProcessor::processObject($context));
         }
     }