Commit Graph

110 Commits

Author SHA1 Message Date
ca08d02f2f Add warning message for import + download images
When import isn't async and downloading images is enabled, we warn the user that import can fail.
Also, added missing translations
2016-11-03 15:59:20 +01:00
e46fdf3096 CS 2016-11-02 07:26:14 +01:00
1f66d79e6b Add more importer to wallabag:import command
All importer available expect Pocket which require an oAuth login.
2016-11-02 07:10:57 +01:00
7816eb622d Add entry.saved event to import & rest 2016-11-02 07:10:23 +01:00
45fd7e09d7 Cleanup 2016-10-30 09:58:53 +01:00
5453500458 Requeue depending on producer
Browser import can requeue message from `parseEntry` but we should take care of the way import are handled (depending on the producer)
2016-10-20 21:17:45 +02:00
1e3d74a9cf Avoid RabbitMQ consumer to loop
When the `parseEntry` returns null it means the entry already exists in the database. Sending `false` as return, will requeue the message which will then loop forever.
2016-10-20 21:17:03 +02:00
c7ea9b41f3 Add controller test for Instapaper 2016-09-27 20:27:08 +02:00
ff1a5362f7 Add Instapaper import
Also update ImportController with latest import (chrome, firefox & instapaper).
2016-09-27 20:27:08 +02:00
d6de23a100 Merge pull request #2192 from wallabag/import-browser-bookmarks
Import Firefox & Chrome bookmarks into wallabag
2016-09-26 14:47:02 +02:00
990adfb34c Move prepareEntry to dedicated place
Yeah first try was ugly, now each part are in the dedicated place.
Also, the date is hardly truncated to 10 chars because Firefox date are 16 chars long and Chrome are 17 chars long. So instead of divised them by a huge number, I prefer to truncate them.
2016-09-26 07:30:02 +02:00
12d93e6896 Update Firefox file
With real data, the previous looks more than a Chrome converted file.
Also, fix date conversion (hope so).
2016-09-25 22:24:07 +02:00
27acc6ddb8 Fix bad date format in Browser import 2016-09-25 15:32:02 +02:00
bd206a84d8 Fixed tests by removing clear() 2016-09-25 12:29:19 +02:00
64b1229b2d fix tests 2016-09-25 12:29:19 +02:00
2c61db30b7 cs & fixes 2016-09-25 12:29:19 +02:00
59201088b4 bring chrome and firefox as separate imports 2016-09-25 12:29:18 +02:00
efe659ab84 Add Chrome path for Mac OS 2016-09-25 12:28:55 +02:00
ae669126e7 Import Firefox & Chrome bookmarks into wallabag 2016-09-25 12:28:54 +02:00
401135852c Use scheduled entity insertions to avoid tag duplicate
Using `getScheduledEntityInsertions()` we can retrieve not yet flushed but already persisted entities and then avoid tags duplication on import.
2016-09-25 12:03:49 +02:00
13a522dfbd Display a message when async import won’t work
Mostly if Redis or RabbitMQ isn’t installed / launched.
I guess it’ll avoid some issues.
2016-09-24 19:56:15 +02:00
59b97fae99 Avoid losing entry when fetching fail
Instead of just say “Failed to save entry” we’ll save the entry at all cost and try to fetch content. If fetching content failed, the entry will still be saved at least, but without content.
2016-09-17 07:40:56 +02:00
fbb319f064 Missing some migrations
and CS
2016-09-16 22:58:33 +02:00
ebe0787e09 Moved Pocket token to user config 2016-09-16 22:22:25 +02:00
4fc998245c Only display message in queue for admin
Instead of for EVERYONE
2016-09-16 20:08:07 +02:00
ac87e0db2a AMPQ -> AMQP 2016-09-14 10:18:02 +02:00
c80cc01afa Change flash message for queued articles 2016-09-13 21:09:05 +02:00
e01a3c98d6 Display how many messages are queue
- update the docker-composer to add Redis
- add migrations
2016-09-13 20:32:52 +02:00
ebf5e5087d Add tests on ImportCommand 2016-09-11 22:15:31 +02:00
015c7a8359 Add more tests
And ability to define how many messages can be hanle by the redis worker before stopping (usefull for tests)
2016-09-11 22:15:31 +02:00
7d862f83b9 Re-facto EntryConsumer
Using an abstract method allow to share code but also can be used it we add a new broker in the future
2016-09-11 22:15:31 +02:00
dc69e25f97 Display a message when async is enabled 2016-09-11 22:15:28 +02:00
b3437d58ae Enable Redis async import
- using javibravo/simpleue
- internal config value are now `import_with_redis` & `import_with_rabbit` which are more clear
- if both option are enable rabbit will be choosen
- services imports related to async are now splitted into 2 files: `redis.yml` & `rabbit.yml`
-
2016-09-11 21:58:56 +02:00
7f7531171f Retrieve created date from Pocket 2016-09-11 21:58:56 +02:00
13470c3596 Add test for RabbitMQ
Also update Symfony deps
2016-09-11 21:58:56 +02:00
8664069e1a Fix DateTime & clear() 2016-09-11 21:58:55 +02:00
6d65c0a8b0 Add ability to define created_at for all import
At the moment only Readability & wallabag v2 import allow created_at import.
Pocket removed `time_added` field from their API v2 to v3...
And wallabag v1 doesn't export that value.
2016-09-11 21:58:55 +02:00
3aca0a9f00 CS 2016-09-11 21:58:55 +02:00
3849a9f323 Some cleanup & refactor 2016-09-11 21:58:55 +02:00
02f6489572 Retrieve all items from Pocket
5000 by 5000.
Also, retrieve newest item first.
2016-09-11 21:58:31 +02:00
c98db1b653 Convert other imports to Rabbit 2016-09-11 21:58:31 +02:00
ef75e1220e Send every imported item to the queue
Instead of queing real Entry to process, we queue all the item to import from Pocket in a raw format.
Then, the worker retrieve that information, find / create the entry and save it.
2016-09-11 21:57:46 +02:00
40d2a29443 Replace RabbitMQ injection with CraueConfiguration 2016-09-11 21:57:09 +02:00
56c778b415 1st draft for rabbitMQ 2016-09-11 21:57:09 +02:00
88e88016b9 Clearing entities in the loop fail on Postgres
It looks like when you clear entities on Postgres some references are lost and tags are not saved :-/
2016-09-11 16:30:01 +02:00
58fadbc9df Fix error on EntityManager clear
Introduced in the recent 2.5.5 release.
Also updated deps.
2016-09-11 15:53:16 +02:00
a1a1077059 Add tests on ReadabilityImport 2016-09-01 13:39:41 +02:00
03e3753f6b Add Readability import
Based on the JSON export instead of the API (which will be shutting down by the September 30, 2016)
2016-09-01 08:00:30 +02:00
79efca1e6f Merge remote-tracking branch 'origin/master' into 2.1 2016-08-22 23:03:16 +02:00
4e4e8e9899 Merge pull request #2221 from wallabag/replace-favorite-star
Replaced favorite word/icon with star one
2016-08-21 22:11:20 +02:00