fix drive issues

This commit is contained in:
Lê Thành Đạt committed 2025-12-06 01:24:45 -05:00
1 parent 2ba360ff54
commit 1779b0291d
7 files changed
+220 -332

No files matched your search

+7
View File
@@ -260,3 +260,10 @@ database/BSK.db-wal
*.db-wal
database/*.db
*.old
*.jar
*.db
*.db-shm
*.db-wal
database/*.db
database/*.db-shm
database/*.db-wal
-304
View File
@@ -1,304 +0,0 @@
# 🗄️ BSK Clinic Database Backup Guide
## ⚠️ **CRITICAL: Understanding SQLite WAL Mode**
Your database uses **WAL (Write-Ahead Logging)** mode for better concurrent access. This creates **3 files**:
```
database/
├── BSK.db ← Main database (updated periodically via "checkpoint")
├── BSK.db-wal ← Write-Ahead Log (contains RECENT uncommitted data)
└── BSK.db-shm ← Shared Memory (coordination file)
```
### **The Problem You Experienced:**
```
❌ WRONG - What Happened:
1. You copied ONLY BSK.db (old data from last checkpoint)
2. You LOST BSK.db-wal (contained checkups 184-1978)
3. Result: 1795 checkups disappeared!
✅ CORRECT - What You Should Do:
1. Stop the server OR checkpoint first
2. Copy the main BSK.db file (now contains all data)
3. Restore it
```
---
## 📋 **Backup Methods**
### **Method 1: Using Built-in Google Drive Backup (RECOMMENDED) ✅**
This is now **COMPLETELY SAFE** - backs up ALL 3 database files!
**Steps:**
1. Open **Server Dashboard**
2. Click **"Sao lưu CSDL lên Drive"** button
3. Watch the progress:
- "Đang kiểm tra các tệp cơ sở dữ liệu..."
- Shows size of each file (BSK.db, BSK.db-wal, BSK.db-shm)
- "Đang tải lên tất cả các tệp database..."
- "✅ Sao lưu hoàn tất!"
4. ✅ Done! All files backed up to a timestamped folder.
**What happens behind the scenes:**
```java
// server.jar automatically does this:
1. Checks all 3 files: BSK.db, BSK.db-wal, BSK.db-shm
2. Creates timestamped folder: "_Database_Backups/Backup_2025-01-15_14-30-00/"
3. Uploads ALL files to that folder
4. No data loss possible!
```
**Google Drive Structure:**
```
BSK_Clinic_Patient_Files/
└── _Database_Backups/
├── Backup_2025-01-15_14-30-00/
│ ├── BSK.db (main database)
│ ├── BSK.db-wal (uncommitted changes)
│ └── BSK.db-shm (shared memory)
└── Backup_2025-01-15_09-15-00/
└── ... (older backup)
```
---
### **Method 2: Manual Backup with PowerShell Script**
**For manual backups when server is running:**
```powershell
# Step 1: Checkpoint the database
.\checkpoint_database.ps1
# Step 2: Copy the database
Copy-Item database\BSK.db "D:\Backups\BSK_backup_$(Get-Date -Format 'yyyy-MM-dd_HH-mm-ss').db"
```
**For manual backups when server is stopped:**
```powershell
# Just copy all 3 files
Copy-Item database\BSK.db* "D:\Backups\"
```
---
### **Method 3: Using SQLite Command Line**
```bash
# Option A: Checkpoint first (if server is running)
sqlite3 database/BSK.db "PRAGMA wal_checkpoint(TRUNCATE);"
# Option B: Create a proper backup
sqlite3 database/BSK.db ".backup 'backup.db'"
```
---
## 🔄 **Restoring from Backup**
### **Method 1: Restore Complete Backup from Google Drive**
**Steps:**
1. **Stop the server** completely
2. Download the **entire backup folder** from Google Drive
3. Navigate to `_Database_Backups/Backup_YYYY-MM-DD_HH-MM-SS/`
4. Download all 3 files:
- `BSK.db`
- `BSK.db-wal` (if present)
- `BSK.db-shm` (if present)
5. **Delete or backup your current database files**:
```powershell
# Backup current database
Move-Item database\BSK.db* backup_old\
```
6. **Copy downloaded files** to `database/` folder
7. **Restart the server**
8. ✅ **Done!** All data restored including uncommitted transactions
### **Method 2: Restore Only Main DB (Data Loss Risk!)**
⚠️ **WARNING:** Only use this if no WAL file exists in the backup!
**Steps:**
1. Stop the server
2. Download only `BSK.db` from backup
3. Copy to `database/` folder
4. Restart server
5. ⚠️ Any data in the WAL file at backup time is **lost**
---
## 🔧 **When to Checkpoint**
SQLite automatically checkpoints when:
- WAL file reaches **1000 pages** (~4MB)
- Database is closed gracefully
- Connection pool is shut down
**Manual checkpoint is needed when:**
- ⚠️ Server has been running for days without restart
- ⚠️ You need to backup while server is running
- ⚠️ WAL file is very large (>10MB)
**Check WAL status:**
```powershell
.\checkpoint_database.ps1 -CheckOnly
```
---
## 🚨 **Emergency: Recovering from Bad Backup**
If you already uploaded a backup that's missing data:
### **Option 1: Check if WAL files still exist at hospital**
```powershell
# At the hospital server, check:
Get-ChildItem database\BSK.db*
# If BSK.db-wal exists:
sqlite3 database\BSK.db "PRAGMA wal_checkpoint(TRUNCATE);"
# Now backup this file - it has all data!
```
### **Option 2: Restore from older backup + replay transactions**
If WAL is gone, data is **permanently lost**. You'll need:
1. Use the latest backup before modification
2. Manually re-enter lost data
3. Use filesystem folder IDs to identify missing checkups:
```powershell
# Find checkup folders
Get-ChildItem image\checkup_media -Directory |
Where-Object { $_.Name -match '^\d+$' } |
Sort-Object { [int]$_.Name }
# Compare with database
sqlite3 database\BSK.db "SELECT checkup_id FROM Checkup ORDER BY checkup_id"
```
---
## 📊 **Best Practices**
### ✅ **DO:**
- ✅ Use the built-in Google Drive backup button
- ✅ Stop the server before manual file copies
- ✅ Test your backups regularly
- ✅ Keep multiple backup versions
- ✅ Backup before schema changes
- ✅ Document any manual database modifications
### ❌ **DON'T:**
- ❌ Copy only BSK.db while server is running
- ❌ Modify the database on your machine while hospital uses it
- ❌ Ignore WAL files during backup
- ❌ Restore old backups without checking timestamps
- ❌ Edit production database without testing first
---
## 🛠️ **Schema Change Workflow**
When you need to add columns or modify the database:
```
CORRECT Process:
──────────────────────────────────────────────────
1. STOP the hospital server completely
2. Backup using: .\checkpoint_database.ps1
3. Download the backup
4. Test changes on your machine
5. Create migration script (ALTER TABLE commands)
6. Send ONLY the migration script (not the whole DB)
7. Run migration script at hospital
8. Restart server
9. Verify everything works
NEVER:
──────────────────────────────────────────────────
❌ Download → Modify → Upload entire database
(This WILL cause data loss!)
```
### **Migration Script Example:**
```sql
-- migration_add_doctor_columns.sql
BEGIN TRANSACTION;
-- Add new column
ALTER TABLE Doctor ADD COLUMN specialty TEXT;
-- Update existing data if needed
UPDATE Doctor SET specialty = 'General' WHERE specialty IS NULL;
COMMIT;
```
**Apply migration:**
```bash
sqlite3 database/BSK.db < migration_add_doctor_columns.sql
```
---
## 🔍 **Troubleshooting**
### **Problem: WAL file is huge (>100MB)**
```powershell
# Force checkpoint
sqlite3 database\BSK.db "PRAGMA wal_checkpoint(TRUNCATE);"
# If it fails, connections are open:
# Stop the server, then retry
```
### **Problem: "database is locked"**
- Server is running → stop it first
- Old connections → restart server
- Check: `database\BSK.db-shm` (if exists, connections are active)
### **Problem: Backup seems too small**
```powershell
# Check WAL size
Get-Item database\BSK.db-wal
# If WAL is large, you forgot to checkpoint!
.\checkpoint_database.ps1
```
---
## 📞 **Support**
If you encounter data loss:
1. **DON'T PANIC**
2. **DON'T overwrite any files**
3. Check if old backups exist on Google Drive
4. Contact: [Your Support Contact]
---
## 📚 **Further Reading**
- [SQLite WAL Mode Documentation](https://www.sqlite.org/wal.html)
- [SQLite Backup Documentation](https://www.sqlite.org/backup.html)
- [Understanding WAL Checkpoints](https://www.sqlite.org/wal.html#ckpt)
---
**Last Updated:** $(Get-Date -Format 'yyyy-MM-dd')
**Version:** 1.0
**Author:** BSK Development Team
Binary file not shown.

After

Width:  |  Height:  |  Size: 5.3 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.3 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.3 KiB

@@ -3379,20 +3379,37 @@ public class CheckUpPage extends JPanel {
private void handleUltrasoundImageDetected(Path imagePath, String patientId) {
log.info("=== HANDLING ULTRASOUND IMAGE: {} ===", imagePath.toAbsolutePath());
log.info("Patient ID for this operation: {}", patientId);
log.info("Current media path: {}", currentCheckupMediaPath);
// Snapshot the current media state atomically to avoid races
final String capturedPatientId;
final Path capturedMediaPath;
synchronized (mediaStateLock) {
capturedPatientId = (patientId == null) ? currentCheckupIdForMedia : patientId;
capturedMediaPath = currentCheckupMediaPath;
}
if (capturedPatientId == null || capturedPatientId.trim().isEmpty() || capturedMediaPath == null) {
log.warn("Media state not ready (patientId or mediaPath null). Skipping: {}", imagePath);
// CRITICAL FIX: Use the patient ID passed as parameter (from folder scan)
// Do NOT fall back to currentCheckupIdForMedia as it may have changed
if (patientId == null || patientId.trim().isEmpty()) {
log.error("CRITICAL: Patient ID is null when processing ultrasound image. This should never happen. Aborting to prevent data corruption.");
return;
}
// Snapshot the current media state atomically to avoid races
final String capturedPatientId = patientId; // Always use the parameter, never currentCheckupIdForMedia
final Path capturedMediaPath;
synchronized (mediaStateLock) {
// Only read the path from shared state, but use the patient ID from parameter
capturedMediaPath = currentCheckupMediaPath;
}
// Validate we have the necessary state
if (capturedMediaPath == null) {
log.warn("Media path is null. This may happen if no patient is currently selected. Skipping: {}", imagePath);
return;
}
// Additional safety check: Ensure the patient ID in the path matches the captured patient ID
String pathPatientId = capturedMediaPath.getFileName().toString();
if (!capturedPatientId.equals(pathPatientId)) {
log.error("CRITICAL RACE CONDITION DETECTED: Patient ID mismatch! Folder scan ID: {}, Current media path ID: {}. Aborting to prevent data corruption.", capturedPatientId, pathPatientId);
return;
}
log.info("Validated patient ID: {} matches current media path", capturedPatientId);
try {
// Final safety check - ensure file still exists before processing
if (!Files.exists(imagePath)) {
@@ -3586,7 +3603,15 @@ public class CheckUpPage extends JPanel {
}
private void handleTakePicture() {
if (currentCheckupMediaPath == null) {
// Atomically capture the media state to prevent race conditions
final String capturedCheckupId;
final Path capturedMediaPath;
synchronized (mediaStateLock) {
capturedCheckupId = currentCheckupIdForMedia;
capturedMediaPath = currentCheckupMediaPath;
}
if (capturedMediaPath == null || capturedCheckupId == null || capturedCheckupId.trim().isEmpty()) {
JOptionPane.showMessageDialog(this, "Vui lòng chọn một lượt khám để lưu ảnh.", "Chưa chọn lượt khám", JOptionPane.WARNING_MESSAGE);
return;
}
@@ -3598,8 +3623,10 @@ public class CheckUpPage extends JPanel {
}
String timestamp = new SimpleDateFormat("yyyyMMdd_HHmmss_SSS").format(new Date());
String fileName = "IMG_" + currentCheckupIdForMedia + "_" + timestamp + ".jpg";
Path filePath = currentCheckupMediaPath.resolve(fileName);
String fileName = "IMG_" + capturedCheckupId + "_" + timestamp + ".jpg";
Path filePath = capturedMediaPath.resolve(fileName);
log.info("Taking picture for checkup ID: {} with filename: {}", capturedCheckupId, fileName);
try {
BufferedImage image = selectedWebcam.getImage();
@@ -3608,11 +3635,12 @@ public class CheckUpPage extends JPanel {
ImageIO.write(image, "JPG", filePath.toFile());
log.info("Picture taken and saved locally as JPG at: {}", filePath);
// Refresh UI to show the new image immediately
loadAndDisplayImages(currentCheckupMediaPath);
// Refresh UI to show the new image immediately (use captured path, not currentCheckupMediaPath)
loadAndDisplayImages(capturedMediaPath);
// Now, upload in the background (server will convert to PNG for archival)
uploadImageInBackground(filePath.getFileName().toString(), image);
// Now, upload in the background
// The filename already contains the checkup ID, so uploadImageInBackground will extract it safely
uploadImageInBackground(fileName, image);
} else {
throw new IOException("Không thể lấy ảnh từ webcam");
@@ -3632,6 +3660,16 @@ public class CheckUpPage extends JPanel {
private void uploadImageInBackground(String fileName, BufferedImage image) {
imageUploadExecutor.submit(() -> {
try {
// CRITICAL FIX: Extract checkup ID from filename instead of using volatile field
// This prevents race conditions when user switches patients during upload
String checkupId = extractCheckupIdFromFilename(fileName);
if (checkupId == null || checkupId.isEmpty()) {
log.error("CRITICAL: Failed to extract checkup ID from filename: {}. Upload aborted to prevent data corruption.", fileName);
return;
}
log.info("Uploading image {} for checkup ID: {}", fileName, checkupId);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
// CHANGE 1: Write the image as a JPG instead of PNG.
@@ -3641,14 +3679,12 @@ public class CheckUpPage extends JPanel {
byte[] imageData = baos.toByteArray();
baos.close();
String checkupId = currentCheckupIdForMedia;
// CHANGE 2: Use the original fileName (which should already be .jpg).
// No need to rename it to .png anymore.
UploadCheckupImageRequest request = new UploadCheckupImageRequest(checkupId, imageData, fileName);
NetworkUtil.sendPacket(ClientHandler.ctx.channel(), request);
log.info("Sent UploadCheckupImageRequest for {}", fileName);
log.info("Sent UploadCheckupImageRequest for {} (checkupId: {})", fileName, checkupId);
} catch (IOException e) {
log.error("Failed to convert image to JPG for upload: {}", fileName, e);
@@ -3656,6 +3692,39 @@ public class CheckUpPage extends JPanel {
});
}
/**
* Extracts checkup ID from filename format: IMG_{checkupId}_{timestamp}.jpg
* This prevents race conditions where currentCheckupIdForMedia changes while processing.
*
* @param fileName The image filename
* @return The checkup ID, or null if extraction fails
*/
private String extractCheckupIdFromFilename(String fileName) {
if (fileName == null || fileName.isEmpty()) {
return null;
}
// Expected format: IMG_{checkupId}_{timestamp}.jpg or ultrasound_{checkupId}_{timestamp}.jpg
try {
String[] parts = fileName.split("_");
if (parts.length >= 2) {
// The checkup ID is always the second part (index 1)
String checkupId = parts[1];
// Validate it's a number
Integer.parseInt(checkupId);
log.debug("Extracted checkup ID '{}' from filename '{}'", checkupId, fileName);
return checkupId;
}
} catch (NumberFormatException e) {
log.warn("Invalid checkup ID format in filename: {}", fileName);
}
log.error("Failed to extract checkup ID from filename: {}", fileName);
return null;
}
private void deleteImage(File imageFile) {
if (imageFile == null || !imageFile.exists()) {
log.warn("Cannot delete image - file does not exist or is null");
@@ -3663,17 +3732,22 @@ public class CheckUpPage extends JPanel {
}
String fileName = imageFile.getName();
String checkupId = currentCheckupIdForMedia;
// CRITICAL FIX: Extract checkup ID from filename instead of using volatile field
// This prevents race conditions when user switches patients during deletion
String checkupId = extractCheckupIdFromFilename(fileName);
if (checkupId == null || checkupId.trim().isEmpty()) {
log.error("Cannot delete image - no checkup ID available");
log.error("Cannot delete image - failed to extract checkup ID from filename: {}", fileName);
JOptionPane.showMessageDialog(this,
"Không thể xóa ảnh - không có mã khám.",
"Không thể xóa ảnh - tên file không hợp lệ: " + fileName,
"Lỗi",
JOptionPane.ERROR_MESSAGE);
return;
}
log.info("Deleting image {} for checkup ID: {}", fileName, checkupId);
// Send delete request to server
DeleteCheckupImageRequest request = new DeleteCheckupImageRequest(checkupId, fileName);
NetworkUtil.sendPacket(ClientHandler.ctx.channel(), request);
@@ -3850,8 +3924,15 @@ public class CheckUpPage extends JPanel {
log.debug("Saved synced image as JPG: {}", imageFile.getAbsolutePath());
// If this image belongs to the currently selected patient, refresh the gallery to show it.
if (response.getCheckupId().equals(currentCheckupIdForMedia)) {
loadAndDisplayImages(currentCheckupMediaPath);
// Use atomic read to prevent race conditions
String currentId;
Path currentPath;
synchronized (mediaStateLock) {
currentId = currentCheckupIdForMedia;
currentPath = currentCheckupMediaPath;
}
if (currentId != null && response.getCheckupId().equals(currentId) && currentPath != null) {
loadAndDisplayImages(currentPath);
}
} catch (IOException e) {
@@ -3866,7 +3947,15 @@ public class CheckUpPage extends JPanel {
private void handleRecordVideo() {
if (currentCheckupMediaPath == null) {
// Atomically capture the media state to prevent race conditions
final String capturedCheckupId;
final Path capturedMediaPath;
synchronized (mediaStateLock) {
capturedCheckupId = currentCheckupIdForMedia;
capturedMediaPath = currentCheckupMediaPath;
}
if (capturedMediaPath == null || capturedCheckupId == null || capturedCheckupId.trim().isEmpty()) {
JOptionPane.showMessageDialog(this, "Vui lòng chọn một lượt khám để lưu video.", "Chưa chọn lượt khám", JOptionPane.WARNING_MESSAGE);
return;
}
@@ -3880,8 +3969,10 @@ public class CheckUpPage extends JPanel {
if (!isRecording) {
// Start recording
String timestamp = new SimpleDateFormat("yyyyMMdd_HHmmss").format(new Date());
String fileName = "VID_" + currentCheckupIdForMedia + "_" + timestamp + ".mp4";
Path filePath = currentCheckupMediaPath.resolve(fileName);
String fileName = "VID_" + capturedCheckupId + "_" + timestamp + ".mp4";
Path filePath = capturedMediaPath.resolve(fileName);
log.info("Starting video recording for checkup ID: {} with filename: {}", capturedCheckupId, fileName);
try {
// Check if JavaCV classes are available
@@ -1266,6 +1266,30 @@ public class ServerHandler extends SimpleChannelInboundHandler<TextWebSocketFram
return;
}
// SECURITY: Validate that filename matches the claimed checkup ID (defense in depth)
String fileNameCheckupId = extractCheckupIdFromFilename(originalFileName);
if (fileNameCheckupId != null && !fileNameCheckupId.equals(checkupId.trim())) {
// CRITICAL SECURITY VIOLATION: Filename contains a DIFFERENT checkup ID than claimed!
log.error("SECURITY VIOLATION: Filename checkup ID '{}' does NOT match claimed checkup ID '{}' for file '{}'",
fileNameCheckupId, checkupId, originalFileName);
log.error("This indicates a potential race condition or malicious upload attempt. Upload REJECTED.");
UserUtil.sendPacket(currentUser.getSessionId(),
new UploadCheckupImageResponse(false,
String.format("Lỗi bảo mật: Tên file không khớp với mã khám (file: %s, claimed: %s). Upload bị từ chối.",
fileNameCheckupId, checkupId),
originalFileName));
return;
}
if (fileNameCheckupId != null) {
log.info("Security check passed: Filename checkup ID '{}' matches claimed checkup ID '{}'",
fileNameCheckupId, checkupId);
} else {
log.debug("Filename '{}' does not contain checkup ID - allowing upload (legacy format or manual upload)",
originalFileName);
}
try {
// 1. Define the storage directory
Path storageDir = Paths.get(Server.imageDbPath, checkupId.trim());
@@ -1308,6 +1332,30 @@ public class ServerHandler extends SimpleChannelInboundHandler<TextWebSocketFram
return;
}
// SECURITY: Validate that filename matches the claimed checkup ID (defense in depth)
String fileNameCheckupId = extractCheckupIdFromFilename(fileName);
if (fileNameCheckupId != null && !fileNameCheckupId.equals(checkupId.trim())) {
// CRITICAL SECURITY VIOLATION: Attempting to delete an image from the WRONG checkup!
log.error("SECURITY VIOLATION: Filename checkup ID '{}' does NOT match claimed checkup ID '{}' for file '{}'",
fileNameCheckupId, checkupId, fileName);
log.error("This could result in deleting images from the wrong patient folder. Delete REJECTED.");
UserUtil.sendPacket(currentUser.getSessionId(),
new DeleteCheckupImageResponse(false,
String.format("Lỗi bảo mật: Tên file không khớp với mã khám (file: %s, claimed: %s). Xóa bị từ chối.",
fileNameCheckupId, checkupId),
fileName));
return;
}
if (fileNameCheckupId != null) {
log.info("Security check passed: Filename checkup ID '{}' matches claimed checkup ID '{}'",
fileNameCheckupId, checkupId);
} else {
log.debug("Filename '{}' does not contain checkup ID - allowing delete (legacy format or manual upload)",
fileName);
}
if (fileName == null || fileName.trim().isEmpty()) {
UserUtil.sendPacket(currentUser.getSessionId(),
new DeleteCheckupImageResponse(false, "Tên file không hợp lệ.", fileName));
@@ -2385,6 +2433,52 @@ public class ServerHandler extends SimpleChannelInboundHandler<TextWebSocketFram
return pattern.toString();
}
/**
* Extracts checkup ID from image filename for security validation.
*
* Expected filename formats:
* - IMG_{checkupId}_{timestamp}.jpg (e.g., IMG_123_20251205_143000.jpg)
* - ultrasound_{checkupId}_{timestamp}.jpg
* - VID_{checkupId}_{timestamp}.mp4
*
* This method is used for server-side validation to ensure the claimed checkup ID
* matches the checkup ID embedded in the filename, preventing race conditions where
* an image might be uploaded to the wrong patient folder due to client-side timing issues.
*
* @param fileName The image/video filename
* @return The checkup ID extracted from the filename, or null if not found or invalid format
*/
private String extractCheckupIdFromFilename(String fileName) {
if (fileName == null || fileName.isEmpty()) {
return null;
}
try {
// Expected format: PREFIX_{checkupId}_{timestamp}.extension
// Split by underscore to extract parts
String[] parts = fileName.split("_");
// Need at least 3 parts: prefix, checkupId, timestamp
if (parts.length >= 2) {
// The checkup ID is always the second part (index 1)
String checkupId = parts[1];
// Validate it's a valid integer (checkup IDs are numeric)
Integer.parseInt(checkupId);
log.debug("Extracted checkup ID '{}' from filename '{}'", checkupId, fileName);
return checkupId;
}
} catch (NumberFormatException e) {
// Filename has underscores but second part is not a number
log.debug("Filename '{}' does not follow expected format (second part is not a valid checkup ID)", fileName);
}
// Return null if filename doesn't follow expected format
// This is OK - allows legacy files or manually uploaded files without checkup ID
return null;
}
/**
* Checks if a character is a Vietnamese vowel or 'd/đ' that can have accents.
* Vietnamese vowels: a, ă, â, e, ê, i, o, ô, ơ, u, ư, y