The scraper was extracting ALL products from the page including
navigation menu items. Now it filters products to only include
those that contain the search query in their name.
Before: Found 47 products (mostly navigation items)
After: Found 2 Capricare products (filtered by query)
- Added PharmacyMap component to ProductView
- Updated backend to accept all valid sources for pharmacies endpoint
- Shows map with pharmacy locations and list of pharmacies with prices
The CIMA search API returns {results: [...]} but the admin panel
was treating it as an array, causing CIMA products to be lost.
Now handles both formats:
- Array: [...]
- Object: {results: [...]}
- ProductView now checks if source is 'cima' to decide endpoint
- All other sources (seed, promofarma, docmorris, primor) use parapharmacy endpoint
- Same fix applied to mobile getProduct function
Before: only 'parapharmacy' source used parapharmacy endpoint
After: all non-CIMA sources use parapharmacy endpoint
Updated validation to accept: cima, promofarma, docmorris, primor, parapharmacy
Before: product_source must be 'cima'
After: product_source must be one of: cima, promofarma, docmorris, primor, parapharmacy
- Updated ProductResults component with new source labels
- Added price display for parapharmacy products
- Updated admin panel to search both CIMA and Parapharmacy
- Updated getSourceBadge to handle parapharmacy sources
- Added CSS for product price display
- Updated ProductView to use correct endpoint for parapharmacy
- Updated getProduct service to handle parapharmacy products
- Shows price, category, brand, source for parapharmacy products
- Updated labels from 'OFF' to 'Parafarmacia'
- Frontend now searches both CIMA and Parapharmacy simultaneously
- Removed filter tabs (Todos/Medicamentos/Parafarmacia)
- Results show in a single unified list
- Mobile: Shows medicines first, then parapharmacy section
- Web: Same unified approach with results summary
- Fixed Primor URL format and selectors
- Created simple N8N webhook that calls scraper API
- All 3 sources now working:
- Promofarma: 20 products
- DocMorris: 10 products
- Primor: 20 products
- Total: 130 products in database
- N8N webhook at /webhook/scrape working correctly
- Added Puppeteer with Chrome in Docker
- Scraper extracts products using data attributes
- Added /api/scrape endpoint
- Tested: 10 products scraped from Promofarma
The scraper now works with Promofarma's HTML structure which uses
data-name, data-pvp attributes on article elements.
- Remove off-service.js and off-service.test.js
- Remove OFF import from server.js
- Simplify /api/products/search to only use CIMA
- Remove 'openfoodfacts' from valid product sources
- Update frontend-mobile Product interface to remove OFF source
- Remove OFF env vars from .env.example
- Add PARAPHARMACY_API_URL to .env.example
OFF was replaced by the new parapharmacy-api microservice.
- Create parapharmacy-api microservice with Express + Swagger + MongoDB
- Add product model with full-text search and deduplication
- Add CRUD endpoints and bulk upsert for scrapers
- Update docker-compose.yml with MongoDB, N8N, and parapharmacy-api services
- Add proxy endpoints in FarmaFinder backend for parapharmacy
- Update frontend services to use parapharmacy API
- Add Swagger documentation at /api/docs
The top-level await redisClient.connect() in redis-client.js was
blocking module import when no Redis server is available (CI env).
Add NODE_ENV guard + reconnect strategy with max retries.
The resolved @opentelemetry/semantic-conventions version in CI does not
export ATTR_SERVICE_NAMESPACE/ATTR_SERVICE_NAME as named exports, breaking
the backend test import. Use plain 'service.name'/'service.namespace'
string keys so the code is independent of the semantic-conventions version.
The top-level await redisClient.connect() in redis-client.js was
blocking module import when no Redis server is available (CI env).
Add NODE_ENV guard + reconnect strategy with max retries.
- Lazy-load tesseract.js only in OCR route (skips 30MB WASM download)
- Conditional tracing.js import (skip OTel gRPC in test mode)
- Remove unused barcode-detector import
- Add testTimeout/openHandlesTimeout to jest.config.js
- Add --forceExitTimeout=30000 to test script
- Split CI test jobs into parallel workflows (backend/frontend/mobile)
- Add timeout-minutes: 15 to all test jobs
- Use npm ci --ignore-scripts + selective native rebuild
The backend test was hanging in CI because server.js imports off-service.js
which pulls in redis-client.js. Without a mock, it tries to connect to Redis
at import time, hanging indefinitely in CI where no Redis server exists.
Also added the missing searchOTC mock for cima-service.js to fix the
SyntaxError from the export mismatch.
- Use Promise.allSettled for parallel API calls
- Reduces total search time from ~1300ms to ~500ms
- Prevents results disappearing during sequential searches
- Create ProductView component with product details display
- Add ProductView.css with responsive styling
- Add product screen to App.jsx screen system
- Pass onNavigateToProduct callback from App to SearchView
- Update ProductResults onSelect to use callback instead of window.location
- READ operations don't require auth per OFF docs (only User-Agent)
- Auth credentials were causing 503 errors
- Keep User-Agent header which is required
- Switch from /cgi/search.pl (503 errors) to /api/v2/search with brand filter
- Add required User-Agent header per OFF documentation
- Add OFF_USER_AGENT env var for customization
- Update .env.example with User-Agent documentation
- Retry logic continues to work with new endpoint
- Add retry logic with exponential backoff (2 retries)
- Remove custom User-Agent that was being blocked by OFF
- Fix syntax error (missing catch block)
- OFF API blocks anonymous users during high demand
- Add OFF_USERNAME and OFF_PASSWORD env vars
- Use Basic Auth when credentials are provided
- Improves rate limits for authenticated requests
- Auth is optional (works without credentials)
- Validate response is JSON before parsing (OFF returns HTML on 503)
- Don't cache empty results to avoid blocking future searches
- Add validateStatus to only accept 200 responses
- Better logging for API failures
- Changed from /api/v2/search (returning 503) to /cgi/search.pl (working)
- Removed strict categories_tags=baby-food filter that was too restrictive
- Fixed test file to use dynamic imports for ES module compatibility
- Increased timeout to 10s for reliability
Antoni Nuñez Romeu