Antoni Nuñez Romeu
30f97fe87d
- Create comprehensive N8N workflow for all 6 parapharmacy sources - Create webhook-triggered manual scraping workflow - Add seed script with 20 sample products - Update n8n/README.md with complete documentation - Update docs/parapharmacy.md with seeding instructions - Add seed script to package.json Sources configured: - Promofarma - Pharmarket - DocMorris - 1001Farma - Primor - MiFarma
388 lines
9.3 KiB
Markdown
388 lines
9.3 KiB
Markdown
# FarmaFinder Parapharmacy System
|
|
|
|
Sistema completo de búsqueda de productos de parafarmacia mediante scraping de múltiples tiendas españolas.
|
|
|
|
## Arquitectura
|
|
|
|
```
|
|
┌─────────────────┐ ┌──────────────────┐ ┌─────────────────┐
|
|
│ N8N Workflows │────▶│ Parapharmacy │────▶│ MongoDB │
|
|
│ (Scraper) │ │ API (Express) │ │ │
|
|
└─────────────────┘ └────────┬─────────┘ └─────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────┐
|
|
│ FarmaFinder │
|
|
│ Backend │
|
|
└─────────────────┘
|
|
```
|
|
|
|
## Componentes
|
|
|
|
| Componente | Puerto | Descripción |
|
|
|------------|--------|-------------|
|
|
| Parapharmacy API | 3002 | API REST para productos de parafarmacia |
|
|
| MongoDB | 27017 | Base de datos de productos |
|
|
| N8N | 5678 | Automatización de workflows y scraping |
|
|
|
|
---
|
|
|
|
## Parapharmacy API
|
|
|
|
### Endpoints
|
|
|
|
| Método | Ruta | Descripción |
|
|
|--------|------|-------------|
|
|
| GET | `/api/products/search?q=term` | Buscar productos (full-text) |
|
|
| GET | `/api/products/:id` | Detalle de producto |
|
|
| GET | `/api/products` | Listar productos (con filtros) |
|
|
| POST | `/api/products` | Crear producto |
|
|
| POST | `/api/products/bulk` | Crear/actualizar múltiples (para scraper) |
|
|
| PUT | `/api/products/:id` | Actualizar producto |
|
|
| DELETE | `/api/products/:id` | Eliminar producto |
|
|
| GET | `/api/products/categories` | Listar categorías |
|
|
| GET | `/api/products/brands` | Listar marcas |
|
|
| GET | `/api/sources` | Fuentes configuradas |
|
|
| GET | `/api/health` | Health check |
|
|
| GET | `/api/docs` | Swagger UI |
|
|
|
|
### Ejemplo de Búsqueda
|
|
|
|
```bash
|
|
# Buscar "capricare"
|
|
curl "http://localhost:3002/api/products/search?q=capricare"
|
|
|
|
# Buscar con filtros
|
|
curl "http://localhost:3002/api/products/search?q=crema&category=dermocosmetica&brand=bioderma"
|
|
```
|
|
|
|
### Respuesta
|
|
|
|
```json
|
|
{
|
|
"results": [
|
|
{
|
|
"_id": "...",
|
|
"name": "Capricare 1 Leche en polvo",
|
|
"brand": "Capricare",
|
|
"category": "Fórmulas lácteas",
|
|
"price": 12.99,
|
|
"original_price": 14.99,
|
|
"image_url": "https://...",
|
|
"source": "promofarma",
|
|
"source_url": "https://promofarma.com/..."
|
|
}
|
|
],
|
|
"total": 5,
|
|
"page": 1,
|
|
"pages": 1
|
|
}
|
|
```
|
|
|
|
### Schema MongoDB
|
|
|
|
```javascript
|
|
{
|
|
name: String, // Nombre del producto
|
|
brand: String, // Marca
|
|
category: String, // Categoría principal
|
|
subcategory: String, // Subcategoría
|
|
description: String, // Descripción
|
|
image_url: String, // URL de la imagen
|
|
source_url: String, // URL en la tienda original
|
|
price: Number, // Precio actual
|
|
original_price: Number, // Precio anterior (si hay descuento)
|
|
currency: String, // EUR por defecto
|
|
source: String, // 'promofarma', 'pharmarket', etc.
|
|
source_product_id: String,
|
|
available: Boolean, // Disponible actualmente
|
|
rating: Number, // Valoración (0-5)
|
|
review_count: Number,
|
|
scraped_at: Date, // Cuándo se scrappeó
|
|
created_at: Date,
|
|
updated_at: Date
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## N8N Configuration
|
|
|
|
### Cuenta de Administrador
|
|
|
|
Al iniciar N8N por primera vez, se crea automáticamente una cuenta de administrador:
|
|
|
|
| Campo | Valor |
|
|
|-------|-------|
|
|
| **Email** | admin@farmafinder.com |
|
|
| **Password** | change-me |
|
|
|
|
> **IMPORTANTE**: Cambia la contraseña después del primer login.
|
|
|
|
### Variables de Entorno (`.env`)
|
|
|
|
```bash
|
|
# N8N Configuration
|
|
N8N_USER=admin
|
|
N8N_PASSWORD=change-me
|
|
N8N_EMAIL=admin@farmafinder.com
|
|
|
|
# Parapharmacy API
|
|
PARAPHARMACY_API_URL=http://parapharmacy-api:3002
|
|
MONGODB_URI=mongodb://mongodb:27017/parapharmacy
|
|
```
|
|
|
|
### Acceso a N8N
|
|
|
|
- **URL**: http://localhost:5678
|
|
- **Email**: admin@farmafinder.com
|
|
- **Password**: change-me
|
|
|
|
---
|
|
|
|
## Workflows de Scraping
|
|
|
|
| Workflow | Trigger | Estado | Descripción |
|
|
|----------|---------|--------|-------------|
|
|
| Parapharmacy Scraper - All Sources | Cada 3 días a las 2am | Inactivo | Scraping automático de las 6 tiendas |
|
|
| Parapharmacy Manual Scraper | POST `/webhook/scrape-all` | Activo | Scraping manual bajo demanda |
|
|
|
|
### 1. Parapharmacy Scraper - All Sources (Automático)
|
|
|
|
- **Trigger**: Cada 3 días a las 2:00 AM
|
|
- **Estado**: Inactivo por defecto
|
|
- **Función**: Scraping automático de las 6 tiendas
|
|
|
|
**Para activar:**
|
|
1. Ir a http://localhost:5678/workflows
|
|
2. Abrir "Parapharmacy Scraper - All Sources"
|
|
3. Hacer clic en "Active" toggle
|
|
|
|
### 2. Parapharmacy Manual Scraper (Webhook)
|
|
|
|
- **Trigger**: POST a `/webhook/scrape-all`
|
|
- **Estado**: Activo por defecto
|
|
- **Función**: Scraping bajo demanda de todas las fuentes
|
|
|
|
**Uso:**
|
|
|
|
```bash
|
|
# Scraping con queries por defecto (todas las tiendas)
|
|
curl -X POST http://localhost:5678/webhook/scrape-all
|
|
|
|
# Scraping con queries específicas
|
|
curl -X POST http://localhost:5678/webhook/scrape-all \
|
|
-H "Content-Type: application/json" \
|
|
-d '{"queries": ["crema hidratante", "protector solar"]}'
|
|
|
|
# Scraping solo de fuentes específicas
|
|
curl -X POST http://localhost:5678/webhook/scrape-all \
|
|
-H "Content-Type: application/json" \
|
|
-d '{"sources": ["promofarma", "pharmarket"]}'
|
|
```
|
|
|
|
---
|
|
|
|
## Fuentes de Scraping
|
|
|
|
| Fuente | URL | Estado |
|
|
|--------|-----|--------|
|
|
| Promofarma | promofarma.com | ✅ Configurado |
|
|
| Pharmarket | pharmarket.es | ✅ Configurado |
|
|
| DocMorris | docmorris.es | ✅ Configurado |
|
|
| 1001Farma | 1001farma.net | ✅ Configurado |
|
|
| Primor | primor.eu | ✅ Configurado |
|
|
| MiFarma | mifarma.es | ✅ Configurado |
|
|
|
|
---
|
|
|
|
## Poblar Base de Datos
|
|
|
|
### Datos de Prueba
|
|
|
|
```bash
|
|
# Ejecutar seed script (inserta 20 productos de prueba)
|
|
cd apps/parapharmacy-api
|
|
npm run seed
|
|
```
|
|
|
|
### Scraping Real
|
|
|
|
```bash
|
|
# Ejecutar scraping manual via N8N webhook
|
|
curl -X POST http://localhost:5678/webhook/scrape-all
|
|
```
|
|
|
|
### Verificar Datos
|
|
|
|
```bash
|
|
# Buscar productos
|
|
curl "http://localhost:3002/api/products/search?q=crema"
|
|
|
|
# Contar productos
|
|
curl "http://localhost:3002/api/products?limit=1"
|
|
# Respuesta: { "total": 20, ... }
|
|
```
|
|
|
|
---
|
|
|
|
## Integración con FarmaFinder Backend
|
|
|
|
El backend principal proxies las peticiones a la API de parafarmacia:
|
|
|
|
### Endpoints Proxy
|
|
|
|
| Método | Ruta | Descripción |
|
|
|--------|------|-------------|
|
|
| GET | `/api/products/parapharmacy/search` | Buscar productos |
|
|
| GET | `/api/products/parapharmacy/:id` | Detalle de producto |
|
|
| GET | `/api/products/parapharmacy/categories` | Categorías |
|
|
| GET | `/api/products/parapharmacy/brands` | Marcas |
|
|
|
|
### Ejemplo desde Frontend
|
|
|
|
```javascript
|
|
// Buscar productos de parafarmacia
|
|
const response = await fetch('/api/products/parapharmacy/search?q=capricare');
|
|
const data = await response.json();
|
|
// data.results = [{ name: "Capricare...", price: 12.99, ... }]
|
|
```
|
|
|
|
---
|
|
|
|
## Docker Setup
|
|
|
|
### Servicios
|
|
|
|
```yaml
|
|
services:
|
|
parapharmacy-api: # Puerto 3002
|
|
mongodb: # Puerto 27017
|
|
n8n: # Puerto 5678
|
|
```
|
|
|
|
### Iniciar
|
|
|
|
```bash
|
|
# Todos los servicios
|
|
docker-compose up -d
|
|
|
|
# Solo parafarmacia
|
|
docker-compose up -d parapharmacy-api mongodb n8n
|
|
|
|
# Ver logs
|
|
docker-compose logs -f parapharmacy-api
|
|
docker-compose logs -f n8n
|
|
```
|
|
|
|
### Detener
|
|
|
|
```bash
|
|
docker-compose down
|
|
```
|
|
|
|
### Limpiar datos
|
|
|
|
```bash
|
|
# Eliminar volumes (borra datos)
|
|
docker-compose down -v
|
|
```
|
|
|
|
---
|
|
|
|
## Troubleshooting
|
|
|
|
### N8N muestra página /setup
|
|
|
|
Verifica que las variables de entorno estén configuradas:
|
|
|
|
```bash
|
|
N8N_OWNER_EMAIL=admin@farmafinder.com
|
|
N8N_OWNER_PASSWORD=change-me
|
|
```
|
|
|
|
### Webhook no funciona
|
|
|
|
1. Verifica que el workflow esté activo en N8N
|
|
2. Revisa el historial de ejecuciones: http://localhost:5678/executions
|
|
3. Revisa logs: `docker logs n8n`
|
|
|
|
### Productos no se guardan
|
|
|
|
1. Verifica que parapharmacy-api esté ejecutándose
|
|
2. Revisa logs: `docker logs parapharmacy-api`
|
|
3. Verifica que MongoDB esté conectado
|
|
4. Prueba el health check: `curl http://localhost:3002/api/health`
|
|
|
|
### MongoDB no conecta
|
|
|
|
```bash
|
|
# Verificar que MongoDB está corriendo
|
|
docker-compose ps mongodb
|
|
|
|
# Ver logs
|
|
docker-compose logs mongodb
|
|
|
|
# Reiniciar
|
|
docker-compose restart mongodb
|
|
```
|
|
|
|
### API devuelve error 500
|
|
|
|
```bash
|
|
# Ver logs de la API
|
|
docker-compose logs parapharmacy-api
|
|
|
|
# Verificar conexión a MongoDB
|
|
docker-compose exec mongodb mongosh --eval "db.adminCommand('ping')"
|
|
```
|
|
|
|
---
|
|
|
|
## Desarrollo
|
|
|
|
### Estructura de Archivos
|
|
|
|
```
|
|
apps/parapharmacy-api/
|
|
├── src/
|
|
│ ├── server.js # Express + Swagger
|
|
│ ├── config.js # Configuración
|
|
│ ├── models/
|
|
│ │ └── Product.js # Schema MongoDB
|
|
│ └── routes/
|
|
│ └── products.js # Endpoints
|
|
├── Dockerfile
|
|
├── package.json
|
|
└── README.md
|
|
|
|
n8n/
|
|
├── workflows/
|
|
│ ├── parapharmacy-scraper.json
|
|
│ └── parapharmacy-manual-scraper.json
|
|
└── README.md
|
|
```
|
|
|
|
### Ejecutar en Desarrollo
|
|
|
|
```bash
|
|
# API
|
|
cd apps/parapharmacy-api
|
|
npm run dev
|
|
|
|
# MongoDB (necesario)
|
|
docker run -d -p 27017:27017 mongo:7
|
|
```
|
|
|
|
### Tests
|
|
|
|
```bash
|
|
cd apps/parapharmacy-api
|
|
npm test
|
|
```
|
|
|
|
### Swagger Docs
|
|
|
|
Acceso a documentación interactiva:
|
|
http://localhost:3002/api/docs
|