f3bfb875e9
Use hash given url to avoid duplicate
...
Using hashed url we can ensure an index on them to ensure it's fast.
2019-05-29 15:56:20 +02:00
b7fa51ae7d
Added given_url in entry table
...
- Added index on entry table for given_url field
- Fix tests:
The previous `bit.ly` url redirected to doc.wallabag but that url doesn't exist in the fixtures.
I used our own internal "redirector" to create a redirect to an url which exist in the fixtures.
Also, updating current migration to use the new `WallabagMigration`.
2019-05-29 13:50:59 +02:00
6e68417f03
Fix tests after rebase
2019-05-28 12:02:17 +02:00
b6c1e1bacc
Fix some tests
2019-05-28 11:44:20 +02:00
5f08426201
Fix because of some breaking changes of Graby 2.0
2019-05-28 11:42:27 +02:00
423efadefc
Set first picture as preview picture
2019-05-21 20:38:22 +02:00
2dbb5b2307
Enable no-referrer on img tags, enable strict-origin-when-cross-origin by default
...
Fixes #3889
Signed-off-by: Kevin Decherf <kevin@kdecherf.com >
2019-05-10 23:07:26 +02:00
8ca858ee73
Fix PHP warning
...
Looks like sometimes (usually from import) the `html` key isn’t available.
2019-03-18 06:23:41 +01:00
1b220426e2
phpcs
...
Signed-off-by: Kevin Decherf <kevin@kdecherf.com >
2018-10-24 22:33:32 +02:00
6059967951
updateOriginUrl: remove 'query string' case from ignore list
...
Two urls with a different query string may refer to two different pages
so keep them both.
Signed-off-by: Kevin Decherf <kevin@kdecherf.com >
2018-10-24 22:27:27 +02:00
44e63667d9
updateOriginUrl: add comment blocks for the parse_url diff check
...
Signed-off-by: Kevin Decherf <kevin@kdecherf.com >
2018-10-24 22:13:03 +02:00
5ba5e22a09
updateOriginUrl: rewrite some if, resolving feedbacks from PR
...
Signed-off-by: Kevin Decherf <kevin@kdecherf.com >
2018-10-24 21:54:09 +02:00
b49c87acf1
ignoreOriginUrl: add initial support of ignore lists
...
Add the ability to specify hosts and patterns lists to ignore the given
entry url and replace it with the fetched content url without touching
to origin_url.
This initial support should be reworked in the following months to move
the hardcoded ignore lists in the database.
Signed-off-by: Kevin Decherf <kevin@kdecherf.com >
2018-10-22 23:42:09 +02:00
fc040c749d
updateOriginUrl: add behavior when diff is fragment and query
...
Signed-off-by: Kevin Decherf <kevin@kdecherf.com >
2018-10-22 23:08:58 +02:00
e07fadea76
Refactor updateOriginUrl to include new behaviors behaviors
...
- Leave origin_url unchanged if difference is an ending slash
- Leave origin_url unchanged if difference is scheme
- Ignore (noop) if difference is query string or fragment
Signed-off-by: Kevin Decherf <kevin@kdecherf.com >
2018-10-22 23:01:16 +02:00
781864b954
ContentProxy: swap entry url to origin_url and set new url according to graby content
...
Closes #3529
Signed-off-by: Kevin Decherf <kevin@kdecherf.com >
2018-10-21 16:15:31 +02:00
4a81360efc
ContentProxy: fix a corner case when entry.url is empty in updateEntry
...
Signed-off-by: Kevin Decherf <kevin@kdecherf.com >
2018-10-21 16:13:20 +02:00
83f1c3274f
Run php-cs-fixer for fixing coding standard issues
2018-09-23 22:20:43 +02:00
7a65c2017b
Override the value of the given parameter ($title) with the (hopefully)
...
correct (to UTF-8) converted PDF title
2018-09-21 13:23:39 +02:00
c01d953292
Add tests for logic
...
Try to translate the title of a PDF from UTF-8 (then UTF-16BE, then WINDOWS-1252) to UTF-8
2018-09-21 13:15:00 +02:00
f80f16dfc8
Try to detect the character encoding in PDFs and try to translate
...
the title from the PDF to UTF-8
2018-09-21 13:15:00 +02:00
8648f0c005
Remove type declaration for PHP 5 compatibility
2018-09-21 13:15:00 +02:00
d76a5a6d60
Bugfix: Sanitize the title of a saved webpage from invalid UTF-8 characters
2018-09-21 13:15:00 +02:00
2a1ceb67b4
php-cs-fixer
...
Signed-off-by: Kevin Decherf <kevin@kdecherf.com >
2018-09-05 14:25:32 +02:00
af29e1bf07
Fix empty title and domain_name when exception is thrown during fetch
...
Add a new helper to set a default title when it's empty:
1/ use basename part of entry's path, if any
2/ or use domain name
Fixes #2053
Signed-off-by: Kevin Decherf <kevin@kdecherf.com >
2017-12-13 22:44:31 +01:00
709e21a3f4
Define storeArticleHeaders false by default
...
Fix tests which must use `$storeArticleHeaders`.
Fix CS
2017-11-21 10:37:36 +01:00
8a21985474
Added internal setting to enable/disable headers storage
2017-11-20 18:47:48 +01:00
3ef055ced3
CS
2017-10-09 16:47:15 +02:00
ff9f89fd23
Add a test for updatePublishedAt
...
To avoid error when a content is re-submitted and it previously add a
published date.
Also, fix the `testPostSameEntry`
2017-07-24 17:07:47 +02:00
b236d3f627
Fix updatePublishedAt on already parsed article's date
2017-07-24 16:39:07 +02:00
c18a2476b6
CS
2017-07-03 13:56:39 +02:00
d0ec2ddd23
Fix validateAndSetPreviewPicture
...
Which wasn't covered by a test!
2017-07-03 13:45:04 +02:00
a05b61159e
Fix PATCH method
...
The PATCH method for the entry should only update what user sent to us and not the whole entry as it was before.
Also, sending tags when patching an entry will now remove all current tags & assocatied new ones.
2017-07-03 13:45:04 +02:00
f808b01692
Add a real configuration for CS-Fixer
2017-07-01 09:52:38 +02:00
41d45c6122
Fix empty language and preview pics
2017-06-12 16:46:33 +02:00
80e49ba7b0
Convert - to _ in language
...
Mostly to increase language supports
2017-06-09 11:42:09 +02:00
42f3bb2c63
Use Locale instead of Language
2017-06-09 11:28:04 +02:00
be54dfe4e6
CS
2017-06-08 21:56:20 +02:00
0d349ea670
Validate language & preview picture fields
...
Instead of saving the value of each field right into the content without any validation, it seems better to validate them.
This might sounds obvious now we say that.
2017-06-08 21:51:46 +02:00
645291e8fe
Add ability to patch an entry with more fields
...
Like when we create an entry, we can now patch an entry with new fields:
- content
- language
- preview_picture
- published_at
- authors
2017-06-02 20:52:49 +02:00
ec97072152
No need to catch that Exception
2017-06-01 11:45:02 +02:00
6acadf8e98
Rewrote code & fix tests
2017-06-01 11:31:45 +02:00
843182c7cf
CS
2017-06-01 09:52:09 +02:00
d5c2cc54b5
Fix tests
2017-06-01 09:49:15 +02:00
d0e9b3d640
Add disableContentUpdate import option
...
This commit also decouples the "import" and "update" functions inside
ContentProxy. If a content array is available, it must be passed to the
new importEntry method.
2017-06-01 09:48:14 +02:00
7aba665e48
Avoid returning objects passed by reference.
...
Objects are always passed by reference, so it doesn't make sense to
return an object which is passed by reference as it will always be the
same object. This change makes the code a bit more readable.
2017-06-01 09:43:01 +02:00
f0378b4d7c
Forced date can now be a timestamp too
...
Add adding more tests for forced content
2017-05-31 14:00:15 +02:00
9e349f08a6
Improve docs
2017-05-31 14:00:15 +02:00
0d6cfb884c
Remove htmlawed and use graby instead
...
Instead of using htmlawed (which is already used in graby) use graby directly (which require some refacto on graby side).
Still needs some tests.
2017-05-31 14:00:15 +02:00
74a75f7d43
Use graby ContentExtractor to clean html
...
It might be better to re-use some graby functionalities to clean html instead of building a new system.
2017-05-31 14:00:15 +02:00
