Commit Graph

61 Commits

Author SHA1 Message Date
b49c87acf1 ignoreOriginUrl: add initial support of ignore lists
Add the ability to specify hosts and patterns lists to ignore the given
entry url and replace it with the fetched content url without touching
to origin_url.

This initial support should be reworked in the following months to move
the hardcoded ignore lists in the database.

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2018-10-22 23:42:09 +02:00
fc040c749d updateOriginUrl: add behavior when diff is fragment and query
Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2018-10-22 23:08:58 +02:00
e07fadea76 Refactor updateOriginUrl to include new behaviors behaviors
- Leave origin_url unchanged if difference is an ending slash
- Leave origin_url unchanged if difference is scheme
- Ignore (noop) if difference is query string or fragment

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2018-10-22 23:01:16 +02:00
781864b954 ContentProxy: swap entry url to origin_url and set new url according to graby content
Closes #3529

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2018-10-21 16:15:31 +02:00
28cc645b93 Run php-cs-fixer for fixing coding standard issues (on ContentProxyTest) 2018-09-23 23:42:05 +02:00
d64139d812 Make helper methods strToHex and hexToStr in ContentProxyTest.php private
to prevent misusage (from outside this class)
2018-09-21 13:31:28 +02:00
c01d953292 Add tests for logic
Try to translate the title of a PDF from UTF-8 (then UTF-16BE, then WINDOWS-1252) to UTF-8
2018-09-21 13:15:00 +02:00
e6f12c0734 More robust srcset image attribute handling
Linked to HTMLawed PR https://github.com/kesar/HTMLawed/pull/17
2018-07-12 14:29:30 +02:00
3fbbe0d9f1 Fix image downloading on null image path 2018-07-05 11:40:51 +02:00
778543311f Fix tests 2018-06-06 17:34:20 +02:00
c15bb5ad72 Fix srcset attribute on images downloaded 2018-06-01 13:49:16 +02:00
5661e8d42a Fix countable in tests 2017-12-18 13:29:34 +01:00
bd91bd5c32 Use namespaced PHPUnit classes 2017-12-18 13:29:33 +01:00
709e21a3f4 Define storeArticleHeaders false by default
Fix tests which must use `$storeArticleHeaders`.
Fix CS
2017-11-21 10:37:36 +01:00
8a21985474 Added internal setting to enable/disable headers storage 2017-11-20 18:47:48 +01:00
c18a2476b6 CS 2017-07-03 13:56:39 +02:00
d0ec2ddd23 Fix validateAndSetPreviewPicture
Which wasn't covered by a test!
2017-07-03 13:45:04 +02:00
a05b61159e Fix PATCH method
The PATCH method for the entry should only update what user sent to us and not the whole entry as it was before.
Also, sending tags when patching an entry will now remove all current tags & assocatied new ones.
2017-07-03 13:45:04 +02:00
38520658ad Fix tests 2017-07-03 07:30:54 +02:00
f808b01692 Add a real configuration for CS-Fixer 2017-07-01 09:52:38 +02:00
80784b782b Merge pull request #2683 from wallabag/credentials-in-db
Store credentials in DB
2017-06-20 16:40:48 +02:00
906424c1b6 Crypt site credential password 2017-06-20 16:03:35 +02:00
41d45c6122 Fix empty language and preview pics 2017-06-12 16:46:33 +02:00
0d349ea670 Validate language & preview picture fields
Instead of saving the value of each field right into the content without any validation, it seems better to validate them.
This might sounds obvious now we say that.
2017-06-08 21:51:46 +02:00
577c0b6dd8 Use an alternative way to detect image
When parsing content to retrieve images to save locally, we only check for the content-type of the image response.
In some case, that value is empty.
Now we’re also checking for the first few bytes of the content as an alternative to detect if it’s an image wallabag can handle.
We might get higher image supports using that alternative method.
2017-06-05 22:54:02 +02:00
a687c8d915 Merge pull request #2708 from jcharaoui/import-disablecontentupdate
Import disableContentUpdate
2017-06-02 11:26:37 +02:00
fcad69a427 Replace images with &
Images with `&` in the path weren’t well replaced because they might be with `&amp;` in the html instead.

Replacing `&` with `&amp;` fix the problem.
2017-06-01 22:50:33 +02:00
6acadf8e98 Rewrote code & fix tests 2017-06-01 11:31:45 +02:00
d5c2cc54b5 Fix tests 2017-06-01 09:49:15 +02:00
d0e9b3d640 Add disableContentUpdate import option
This commit also decouples the "import" and "update" functions inside
ContentProxy. If a content array is available, it must be passed to the
new importEntry method.
2017-06-01 09:48:14 +02:00
7aba665e48 Avoid returning objects passed by reference.
Objects are always passed by reference, so it doesn't make sense to
return an object which is passed by reference as it will always be the
same object. This change makes the code a bit more readable.
2017-06-01 09:43:01 +02:00
695af58857 Fix tests following rebase 2017-05-31 14:00:15 +02:00
38a04dee34 CS 2017-05-31 14:00:15 +02:00
f0378b4d7c Forced date can now be a timestamp too
Add adding more tests for forced content
2017-05-31 14:00:15 +02:00
0d6cfb884c Remove htmlawed and use graby instead
Instead of using htmlawed (which is already used in graby) use graby directly (which require some refacto on graby side).
Still needs some tests.
2017-05-31 14:00:15 +02:00
74a75f7d43 Use graby ContentExtractor to clean html
It might be better to re-use some graby functionalities to clean html instead of building a new system.
2017-05-31 14:00:15 +02:00
4423b88c5b Merge pull request #3168 from wallabag/instapaper-tags-import
Add support for tag in Instapaper import
2017-05-31 12:32:09 +02:00
bad7df8c00 CS & improve tags assertions 2017-05-31 10:38:15 +02:00
5dbf3f2326 TagController: ignore ActionMarkAsRead when removing tag from entry
Fixes #2835

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2017-05-31 00:36:46 +02:00
d181bd7285 Merge remote-tracking branch 'origin/master' into 2.3 2017-05-30 09:59:06 +02:00
de8d2a9005 CS
Signed-off-by: Thomas Citharel <tcit@tcit.fr>
2017-05-28 12:59:48 +02:00
6bc6fb1f60 Move Tags assigner to a separate file
Signed-off-by: Thomas Citharel <tcit@tcit.fr>
2017-05-27 22:08:14 +02:00
ac1509a66b Fix documentation URL for "How can I help to fix that?" 2017-05-24 12:57:46 +02:00
a1146b6551 Added tests on logs records 2017-05-12 15:01:18 +02:00
3554364bed Fixed tests 2017-05-12 13:47:53 +02:00
d09fe4d233 Added test for deduplication 2017-05-05 14:33:36 +02:00
3d71d40349 Avoid false preview image
If the website doesn't provide an og_image, the value will be false and so it'll be saved like that in the database.
We prefer to leave it as null instead of false.
2017-01-10 17:42:36 +01:00
a2c1b94e82 Revert switch to KernelTestCase for ContentProxyTest
Define the error string manually inside the test class instead of
fetching it from app config.
2016-12-03 09:50:36 -05:00
fc2b7bda53 Fix ContentProxy tests 2016-12-03 00:14:06 -05:00
10b3509757 Added http_status in Entry entity 2016-11-18 15:09:21 +01:00